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ABSTRACT 



The Third International Mathematics and Science Study 
(TIMSS) covered five different grade levels, with more than 40 countries 
collecting data in more than 30 different J :*' v . ' ? r 4 1 lion 

students were tested. The present report contains the TIMSS results for 
students in the final year of secondary school. Mathematics and science 
literacy achievement results are reported for 21 countries; advanced 
mathematics results and physics results, respectively, are reported for 16 
countries. These results complete the first round of descriptive reports from 
the TIMSS study. Together with the results for primary school students (third 
and fourth grade in most countries) and middle school students (seventh and 
eighth grades in most countries) , the results contained in this report 
provide valuable information about the relative effectiveness of a country's 
education system as students progress through school. A ten-page Executive 
Summary details the extensive conclusions to be drawn from the study. Dozens 
of tables and figures provide detailed statistics for all participating 
countries. The Netherlands and Sweden were the top performing countries in 
mathematics; France was the top performer in advanced mathematics; Norway and 
Sweden had physics achievement levels significantly higher than other 
participating countries. The appendixes contain extensive information 
pertaining to the development of the TIMSS tests, sample sizes and 
participation rates, compliance with sampling guidelines, and the 
test-curriculum matching analysis. (DDR) 
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i— Executive Summary 



Mathematics and Science Achievement in the 
Final Year of Secondary School 



Since its inception in 1959, the International Association for the Evaluation of 
Educational Achievement (IEA) has conducted a series of international comparative 
studies designed to provide policy makers, educators, researchers, and practitioners 
with information about educational achievement and learning contexts. The Third 
International Mathematics and Science Study (TIMSS) is the largest and most 
ambitious of these studies. 

The scope and complexity of TIMSS is enormous. The mathematics and science 
testing covered five different grade levels, with more than 40 countries collecting 
data in more than 30 different languages. More than half a million students were 
tested around the world. The present report contains the results for students in the 
final year of secondary school. 

As can be imagined, testing this “grade” was a special challenge for TIMSS. The 
24 countries participating in this component of the testing vary greatly with respect 
to the nature of their upper secondary education systems. First, there was the 
question of how many students of the age-eligible cohort are even in school by the 
final year, and how this might differ across countries. Second, it was no small task 
for many countries to describe the final year of school. In most TIMSS countries, 
students’ final year of school depends on their course of study (e.g., academic, 
technical, or apprenticeship). Thus, the final year of schooling varies across and 
within countries, with some students completing secondary school after a two-, 
three-, four-, or even five-year program. Understandably, it was difficult for some 
countries to test all of the final-year students, particularly the ones in on-site 
occupational training. To give some indication of the proportion of the entire 
school-leaving age cohort that was covered by the testing in each country, TIMSS 
developed its own index - the TIMSS Coverage Index or TCI. In general, the 
smaller the TCI, the more elite the group of students tested. 

Given the extensive diversity of students’ curricula there also were many questions 
about what mathematics and science understandings students should have to meet 
the challenges beyond secondary school. Thus, TIMSS developed three different 
tests. The mathematics and science literacy test was designed for all final-year 
students, regardless of their school curriculum. By and large, the purpose of this test 
was to measure how well students can use their knowledge in addressing real-world 
problems having a mathematics or science component. This test was designed to 
be reported separately for mathematics and for science. There also was great interest 
on the part of some TIMSS countries to determine what school-leaving students 
with special preparation in mathematics and science know and can do, since the 
capabilities of these students may help determine a country’s future potential to 
compete in a global economy. Thus, a second test was developed for students 
having taken advanced mathematics. For the sciences, it was not possible to study 
all branches of science in detail. The participating countries chose physics for 
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detailed study because it is the branch of science most closely associated with 
mathematics, and came closest to embodying the essential elements of natural 
science. The third test, then, was a physics test designed to measure learning of 
physics concepts and knowledge among final-year students having studied physics. 
Each of the three tests contains multiple-choice questions as well as questions in an 
open-ended format requiring students to generate and write their answers. These 
types of questions, some of which required extended responses, were allotted 
approximately one-third of the testing time. Not all of the 24 countries participated 
in the three different parts of the testing (see Table 1). 

The success of TIMSS depended on a collaborative effort between the research 
centers in each country responsible for implementing the steps of the project and 
the network of centers responsible for managing the across-country tasks such as 
training country representatives in standardized procedures, selecting comparable 
samples of schools and students, and conducting the various steps required for 
data processing and analysis. Most countries tested the mathematics and science 
achievement of their students in May and June of 1995. 

TIMSS was conducted with attention to quality at every step of the way. Rigorous 
procedures were designed specifically to translate the tests, and numerous regional 
training sessions were held in data collection and scoring procedures. Quality 
control monitors observed testing sessions, and sent reports back to the TIMSS 
International Study Center at Boston College. The samples of students selected for 
testing were scrutinized according to rigorous standards designed to prevent bias 
and ensure comparability. In this publication, the countries are grouped for reporting 
of achievement according to their compliance with the sampling guidelines and the 
level of their participation rates. Prior to analysis, the data from each country were 
subjected to exhaustive checks for adherence to the international formats as well as 
for within-country consistency and comparability across countries. 

The results for the students in their final year of secondary school complete the first 
round of descriptive reports from the TIMSS study. Together with the results for 
primary school students (third and fourth grades in most countries) and middle school 
students (seventh and eighth grades in most countries), the results contained herein 
will provide valuable information about the relative effectiveness of a country’s 
education system as students progress through school. 

The following sections summarize the major findings described in this report. 
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Mathematics and Science Literacy 

The report presents mathematics and science literacy achievement results for 21 
countries. Even though there was quite a range in the TCIs, about half the countries 
were able to cover 70% or more of the entire school-leaving age cohort (see Table 1.1). 
Also, contrary to some previous international studies, for the mathematics and 
science literacy testing, the higher-performing countries tended to have better 
coverage than the lower-performing countries. Although differing levels of selectivity 
among education systems was not a large issue, low student participation rates were 
a problem in many of the countries. Because final-year students have many demands 
on their time and their educational situations can make testing difficult (e.g., 
apprenticeship training), countries had some difficulty in encouraging students to 
attend the testing sessions. Only eight countries met the TIMSS guidelines for 
sample participation (see Table 1.1). 

O The Netherlands and Sweden were the top-performing countries. Iceland, 
Norway, and Switzerland also performed well, similar to each other but 
significantly below the Netherlands and Sweden. Other countries 
performing above the international average of the 21 countries were 
Denmark, Canada, New Zealand, and Austria. [The only two high- 
performing countries with a low degree of coverage of the school- 
leaving age cohort (less than 60%) were Denmark and Iceland. However, 
of the high-performing countries, only Sweden, Switzerland, and New 
Zealand met the sampling guidelines. The Netherlands and Denmark 
deviated from the approved sampling procedures and had low 
participation rates.] 

[> Countries performing below the international average were (in descending 
order of average achievement): Hungary, the Russian Federation, Italy, 
the United States, Lithuania, Cyprus, and South Africa. In general, 
Hungary, the Russian Federation, Italy, the United States, and Lithuania 
performed similarly, followed by Cyprus and South Africa. 

[> As noted above, selectivity in education systems and sampling 
approaches did not seem to be much of a factor in the mathematics and 
science literacy testing. Still, to place countries on a more equal footing, 
it is interesting to look at performance for the top 25% of the students in 
the entire school-leaving age cohort. From this perspective, Sweden, the 
Netherlands, Norway, and Switzerland were the highest performing 
countries. 

\> When the results were looked at separately for mathematics and science, 
the top-performers in mathematics literacy were the Netherlands, Sweden, 
Denmark, and Switzerland. The top-performers in science literacy were 
Sweden, the Netherlands, Iceland, and Norway. 
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D> Countries that had higher achievement in mathematics literacy than 
in science literacy were Denmark, France, Hungary, Lithuania, and 
Switzerland. Those with higher achievement in science literacy were 
Canada, the Czech Republic, Iceland, Norway, the Russian Federation, 
Sweden, and the United States. 

t> In all countries except South Africa, males had significantly higher 
average achievement than females in mathematics and science literacy. 
This also was true for science literacy. In mathematics literacy, there 
were no significant gender differences in performance in Hungary, 
the United States, and South Africa. 

t> Countries ranking high in mathematics achievement at the eighth grade 
did not always rank high in mathematics literacy at the upper secondary 
level. Only five countries were above the international average both at 
the eighth grade and for their upper secondary school students: Switzerland, 
the Netherlands, Austria, France, and Canada. 

!> In general, the students no longer taking mathematics performed less 
well in mathematics literacy than those still studying the subject. 
Similarly, there was a positive association between taking science 
subjects and performance in science literacy in almost every country. 

t> In nine countries (Australia, Cyprus, the Czech Republic, France, Hungary, 
Italy, Lithuania, the Russian Federation, and Slovenia), 85% or more of 
the students reported that they were currently taking mathematics. In 
contrast, countries where as many as one-third of the final-year students 
reported that they were not currently taking mathematics included 
Canada, Iceland, the Netherlands, Switzerland, and the United States. 

!> Compared with mathematics, higher percentages of students in most 
countries reported that they were taking no science subject at the time of 
testing. Half or more of the students in the Czech Republic, Denmark, 
Norway, Sweden, and Switzerland, reported that they were not taking 
science, and nearly half of the final-year students so reported in Canada 
and the United States. 

t> Even though a strictly comparable classification of educational programs 
was not always possible across countries, students enrolled in academic 
programs had higher average achievement than students in vocational 
programs. The average achievement of students in technical programs 
generally was somewhere between that of the academic and vocational 
students. 
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> Students generally reported positive perceptions about their performance 
in mathematics and science. The highest perceptions of success in 
mathematics were reported in Australia, Denmark, Italy, and the United 
States, where 70% or more of the students agreed that they usually did 
well. Perceptions of doing well in science were generally higher; in 12 
countries more than 70% of the students agreed that they usually did 
well. Eighty percent or more so agreed in Italy, Lithuania, and the 
United States. 

> Despite the different educational approaches, structures, and organizations 
across the TIMSS countries, it is clear that parents’ education is positively 
related to students’ mathematics and science literacy. As was the case for 
eighth graders, in every country final-year students whose parents had 
more education had higher mathematics and science literacy. 

> More than 30% of students in Canada, Iceland, Lithuania, the Russian 
Federation, and the United States indicated that at least one parent had 
finished university, while in contrast, more than 30% of the students in 
Australia, Cyprus, the Czech Republic, France, Italy, and South Africa 
reported that the highest level attained by either parent was finished 
primary but not upper secondary school. 

O In most countries, more than 80% of the students reported at least weekly 
use of calculators (at school, at home, or anywhere else). Only in the 
Czech Republic, Norway, and the Russian Federation did 20% or more 
of the students report rarely or never using calculators. The frequent use 
of calculators was positively related to mathematics and science literacy 
in all countries. 

O Final-year students were given the option of using a calculator when 
completing the TIMSS tests. Most students made moderate use of a 
calculator on the mathematics and science literacy test. The students 
who reported the most calculator use on the test performed best. 

> The final-year students in a number of countries reported relatively 
infrequent computer use (at school, at home or anywhere else). Only in 
Australia, Austria, Canada, Denmark, Iceland, the Netherlands, New 
Zealand, Switzerland, and the United States did more than 50% of the 
students report at least weekly use of computers. 

[> Students in most countries reported spending between two and three 
hours per day on homework, on average. One-fourth or more of the 
final-year students in Austria, the Czech Republic, the Netherlands, 
Norway, Sweden, Switzerland, and the United States reported studying 
for less than one hour per day. 
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t> Students were also asked about other ways they could spend their time out 
of school. Socializing is clearly an important activity for final-year students, 
with students in many countries devoting up to about two and one-half 
hours each day to spending time with friends. Watching television or 
videos also is a frequent activity (about an hour or so a day). 

t> Students’ reports about the time spent working at a paid job varied across 
countries. In about half the countries, most final-year students (more 
than 80%) reported working at a paid job for less than one hour each day. 
However, in Australia, Canada, Iceland, the Netherlands, New Zealand, 
Norway, and the United States, at least one-fourth of the students reported 
working for three hours or more each day. 



Advanced Mathematics 

The report presents results for 16 countries participating in the testing of students 
having taken advanced mathematics courses. The test questions covered primarily 
the content areas of equations and functions, calculus, and geometry, and results are 
provided overall as well as separately for these three areas. The percentages of 
students tested in each country reflect the fact that a relatively small subset of the 
final-year students in each country have taken the advanced mathematics courses 
necessary to participate in this portion of the testing. The percentages of the school- 
leaving age cohort covered by the sample of students tested in advanced mathematics 
in each country ranged dramatically, although most countries tested 20% or less of 
this cohort. Countries with coverage below 10% were the Russian Federation (2%), 
Lithuania (3%), and Cyprus (9%). Austria (33%) and particularly Slovenia (75%) 
were at the high end. Compared to the mathematics and science literacy testing, 
countries had more success in locating these advanced students and encouraging 
them to participate in the testing. Thus, 10 of the 16 countries met the TIMSS 
sampling guidelines (see Table 5.1). 

t> Led by France, the countries performing above the international average 
of the 16 countries also included the Russian Federation, Switzerland, 
Denmark, Cyprus, and Lithuania. Australia, Greece, Sweden, and 
Canada also performed similarly to several countries in this top group. 
[Among these countries, the Russian Federation and Lithuania tested a 
very small percentage (2-3%) of their school-leaving age cohort. Denmark 
did not meet the TIMSS guidelines for either sampling procedures or 
participation rates, and Australia had school participation rates below 
the required 85%.] 

t> The cluster of lower-performing countries included Slovenia, Italy, the 
Czech Republic, Germany, the United States, and Austria. All except 
Slovenia and Italy performed below the international average. 
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C> Interestingly, looking at the top 10% of the school-leaving age cohort, 
Slovenia and France had significantly higher performance than other 
participating countries. Even though Slovenia had difficulty in imple- 
menting the TIMSS sampling guidelines, the advanced mathematics 
testing covered three-fourths of its entire school-leaving age cohort. 
Similarly, France followed all of the sampling guidelines and also had 
relatively high coverage (20%). It appears that having higher percentages 
of students enrolled in advanced mathematics courses need not have a 
negative impact on the performance of the top students in that group. 

C> Significant gender differences favoring males in advanced mathematics 
achievement were found in all countries except Greece, Cyprus, Australia, 
Italy, and Slovenia. In some countries many more males than females have 
taken advanced mathematics courses, but this varied across countries. 



> Compared to the other participating countries, most countries showed 
particular strengths or weaknesses in the content areas tested. For example, 
Sweden performed above the international average in numbers and 
equations, below the international average in calculus, and about at the 
international average in geometry. 

t> Most countries also did relatively better in some content areas than others 
compared to their overall performance in advanced mathematics. For 
example, compared to their overall average achievement, students in the 
United States performed better in numbers and equations and worse in 
geometry. 

t> Although the majority of students in many TIMSS countries reported 
receiving from three to five hours of mathematics instruction each week, 
in Austria and Sweden more than 60% of the students had less than three 
hours each week, and in Australia, Canada, Cyprus, France, Greece, and 
the Russian Federation, the majority of students had five hours or more. 



O The amount of homework assigned also varied considerably. At one 
extreme, more the 40% of the advanced mathematics students in the 
Czech Republic and Sweden reported that they were assigned mathematics 
homework less than once a week, while at the other extreme, more than 
80% of the students in Australia, Canada, Cyprus, Greece, Lithuania, the 
Russian Federation, and the United States reported having homework 
assigned three or more times a week. 
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t> Advanced mathematics students were asked how often several different 
types of instructional activities were used in their classrooms. Among 
these, almost all students in all countries reported being asked to do 
reasoning tasks in at least some lessons. In almost every country, the 
students with the highest achievement were those that reported engaging 
in reasoning tasks most frequently. 
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O Algebra is an essential component of mathematics in upper secondary 
school, and students in every country reported that they are often asked 
to solve equations in mathematics class. Spending time working on 
equations also was an indicator of high achievement on the TIMSS 
advanced mathematics test. 

C> Final-year advanced mathematics students reported that the use of 
computers to do exercises or solve problems in mathematics class is 
comparatively rare. 

t> Calculator use by final-year advanced mathematics students was very 
common. In Australia, Canada, Cyprus, Denmark, Sweden, and the 
United States, more than 80% of the students reported using a calculator 
daily (at home, at school, or anywhere else), and in several other 
countries more than half of the students reported this level of use. In 
general, the advanced mathematics students with the highest average 
achievement were those who reported the highest level of calculator use. 

t> Most of the advanced mathematics students made moderate use of a 
calculator on the TIMSS test. In general, the students who reported that 
they did not use a calculator on the advanced mathematics test did not 
do as well as those who reported using one, although the extent of 
calculator use was not consistently related to achievement in every country. 

Among the final-year students taking advanced mathematics, the majority 
in every country reported that they plan to attend university. When asked 
about their plans for areas of future study, the most popular choices 
were business, health sciences or related occupations, and engineering. 

D> Even though not many students chose mathematics as their preferred 
area of future study, the majority of the students in many of the countries 
agreed that they would like a job that involved using mathematics. In 
general, more males than females agreed that they would like a job 
involving mathematics. 
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Physics 

Physics achievement results for students having taken physics are reported for 16 
countries. The physics test was designed to measure five content areas: mechanics; 
electricity and magnetism; heat; wave phenomena; and modem physics - particle, 
quantum and astrophysics, and relativity. The percentage of the entire school-leaving 
age cohort that participated in the physics study was approximately 15% in several 
countries, although it varied from as little as 2% to 3% in the Russian Federation, 
Latvia (LSS), and Denmark to 33% in Austria and 39% in Slovenia. Eleven of the 
countries met the TIMSS sampling guidelines (see Table 8.1). 

O Norway and Sweden had average physics achievement similar to each 
other and significantly higher than the other participating countries. The 
Russian Federation and Denmark also performed above the international 
average. [The Russian Federation had a very low coverage index (2%) 
as did Denmark (3%), and Denmark did not comply with the guidelines 
for sampling procedures or participation rates.] 

[> The cluster of lowest-performing countries included France, the Czech 
Republic, Austria, and the United States, all of which performed below 
the international average of the 16 countries. 

[> The country rankings for the top 10% of the school-leaving age cohort 
were quite consistent with those obtained from all the tested students. 
However, the countries most likely to improve their standing were those 
with the largest coverage index, since they were least likely to have 
tested just the elite students. Slovenia joined Sweden as a top-performer, 
despite having difficulties with low sampling participation and unapproved 
sampling procedures. Austria also moved from the lowest-scoring cluster 
of countries to the middle group. 

[> Males had significantly higher physics achievement than females in all 
but one of the participating countries (Latvia (LSS)). Although the 
proportions of males and females taking physics were about equal in 
Latvia (LSS), Canada, the Russian Federation, Switzerland, and the 
United States, in several countries males outnumbered females by two 
or three to one. 

[> Norway and Sweden performed above the international average in all 
five physics content areas, while Austria and the United States fell 
below the international average in all five. Nearly every other country 
scored significantly above or below the international average in at least 
one content area, and about average in the others. 

Compared to their overall physics performance, most countries did 
relatively better in some content areas than others. For example, students 
in Canada performed relatively less well in mechanics and relatively 
better in heat than they did on the physics test as a whole. 

25 
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D> Significant gender differences favoring males were found in more 
countries in the areas of mechanics (15 countries), wave phenomena 
(11 countries), and modem physics (12 countries) than in electricity and 
magnetism (8 countries) or heat (7 countries). 

S> The amount of physics instruction received by students varied considerably 
across countries, but in general was less than five hours per week. The 
assignment of homework also varied considerably from less than once a 
week in several countries to three or more times a week in others. 

O Although laboratory work might be expected to play a central role in 
physics classes, students reports varied across countries. In Austria, 
Germany, and Greece, the majority of the students reported that they 
never or almost never conduct laboratory experiments, whereas one-fourth 
or more of the students in Canada, Cyprus, Denmark, France, Switzerland, 
and the United States reported conducting experiments in most or all 
lessons. In about half the countries, the majority of students reported 
conducting experiments in some lessons. There was no consistent 
relationship between frequency of conducting laboratory experiments in 
class and physics achievement. 

D> Paralleling the findings for advanced mathematics, physics students 
frequently use calculators. Although the relationship was less pronounced 
than for students having taken advanced mathematics, in most countries 
students who reported daily calculator use performed better on the TIMSS 
physics test than those who reported less frequent use. 

> Students were given the option of using a calculator when completing 
the physics test, and most physics students in every country used the 
calculator on some questions. The extent of calculator use was not 
consistently related to achievement in every country, but physics 
students who reported that they did not use a calculator on the test did 
less well than those who reported using one. 

D> Like the plans for further education of final-year students having taken 
advanced mathematics, those of final-year physics students center mainly 
on university. Students who have studied physics are well positioned to 
continue their education in the sciences or in areas of scientific application. 
Although choice of future study area varied considerably across countries, 
the most popular were engineering, mathematics or computer/information 
sciences, health sciences or related occupations, and business. While 
more females than males chose health sciences or related occupations, 
males often outnumbered females by a substantial margin in engineering, 
and in mathematics or computer/information sciences. 
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r - Introduction 



Mathematics and Science Achievement in the 
Final year of Secondary School 



Several major educational issues are addressed by the secondary school assessment 
conducted as part of the Third International Mathematics and Science Study (TIMSS). 
One such issue is how effective educational systems around the world have been 
in educating their whole populations rather than just an elite group of students. 
Given the importance of an understanding of mathematics and science to social and 
economic participation in a technology-based society, there is particular interest 
in what students finishing secondary school know and can do in mathematics and 
science; that is, after studying mathematics and science during their years as students, 
how literate are they in these subjects? 

There is also special interest in what school-leaving students with special prepara- 
tion in advanced mathematics and physics, the potential future mathematics and 
science specialists, know and can do in these subjects. The achievement of these 
students may indicate the ability of countries to compete in a global economy based 
on scientific discoveries, state-of-the-art approaches to financing, and innovations 
in electronics, computing applications, and fast-paced communication technologies. 

Both for the overall school population and for students having taken advanced 
mathematics and physics, the TIMSS data for final-year students can be used to 
help determine what understanding of mathematics and science concepts students 
have after completing their upper secondary schooling, and how effectively they 
might use that understanding as they move on to their future endeavors in school, 
occupational, and community settings. Beyond providing the participating countries 
with a solid basis for examining their students’ performance from an international 
perspective, TIMSS gives each of them an impetus for scrutinizing the quality and 
effectiveness of its education system. 

Together with the previously released results in mathematics and science achievement 
for primary and middle school students, the TIMSS results for students in the final 
year of secondary school can heighten countries’ awareness of a myriad of educational 
issues. By expanding each country’s knowledge of what is possible through learning 
about the achievements of others and the techniques they use, TIMSS affords the 
participants unprecedented opportunity to consider the most-needed reforms and to 
gamer public support for improving students’ learning in mathematics and science. 

TIMSS is the most ambitious and complex comparative education study in a series 
of such undertakings conducted during the past 37 years by the International 
Association for the Evaluation of Educational Achievement (IEA). 1 The main 
purpose of TIMSS was to focus on educational policies, practices, and outcomes 




1 The previous IEA mathematics studies were conducted in 1964 and 1980-82, and the science studies in 
1970-71 and 1983-84. For information about TIMSS procedures, see Appendix B. 
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in order to enhance mathematics and science learning within and across systems of 
education. With its wealth of information from more than half a million students at 
five grade levels in 15,000 schools and 41 countries, TIMSS enables the participants to 
examine similarities and differences in how mathematics and science education 
works and how well it works. The study used innovative testing approaches and 
collected extensive information about the contexts within which students learn 
mathematics and science. 

All countries that participated in TIMSS were to test students in the two grades with 
the largest proportion of 13-year-olds (seventh and eighth grades in most countries) 
in both mathematics and science. Many TIMSS countries also tested the mathematics 
and science achievement of students in the two grades with the largest proportion of 
9-year-olds (third and fourth grades in most countries) and of students in their final 
year of secondary education. Subsets of students in the fourth and eighth grades also 
had the opportunity to participate in a “hands-on” performance assessment. 

Together with the achievement tests, TIMSS administered a broad array of background 
questionnaires. The data collected from students, teachers, and school principals, as 
well as the system-level information collected from the participating countries, 
provide an abundance of information for further study and research. TIMSS data 
make it possible to examine differences in current levels of performance in relation 
to a wide variety of variables associated with the classroom, school, and national 
contexts within which education takes place. 

The results of the assessments of primary and middle school students have been 
published in: 

Mathematics Achievement in the Primary School Years: IEA } s Third International 
Mathematics and Science Study 2 

Science Achievement in the Primary School Years: IEA’s Third International 
Mathematics and Science Study 3 

Mathematics Achievement in the Middle School Years: IEA’s Third International 
Mathematics and Science Study 4 



2 Mullis, I.V.S., Martin, M.O., Beaton, A.E., Gonzalez, E.J., Kelly, D.L, and Smith, T.A. (1997). Mathematics 
Achievement in the Primary School Years: lEA's Third International Mathematics and Science Study ( TIMSS j. 
Chestnut Hill, MA: Boston College. 

3 Martin, M.O., Mullis, I.V.S., Beaton, A.E., Gonzalez, E.J., Smith, T.A., and Kelly, D.L (1997]. Science 
Achievement in the Primary School Years: lEA's Third International Mathematics and Science Study ( TIMSS j. 
Chestnut Hill, MA: Boston College. 

4 Beaton, A.E., Mullis, I.V.S., Martin, M.O., Gonzalez, E.J., Kelly, D.L., and Smith, T.A. (1996). Mathematics 
Achievement in the Middle School Years: lEA's Third International Mathematics and Science Study (TIMSSJ. 
Chestnut Hill, MA: Boston College. 
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Science Achievement in the Middle School Years: lEA’s Third International 
Mathematics and Science Study 5 

Performance Assessment in lEA’s Third International Mathematics and 
Science Study 6 

These reports have been widely disseminated and are available on the internet 
(http://wwwcsteep.bc.edu/timss). The entire TLMSS international database containing 
the achievement and background data underlying these reports also has been released 
and is available at the TIMSS website. 

The present report focuses on the mathematics and science literacy of all students in 
their final year of upper secondary school, and on the advanced mathematics and 
physics achievement of final-year students who have taken advanced courses in 
those subjects. The TIMSS International Study Center also plans to make the data 
collected in the final-year assessment available at its website, together with this report. 



What Assessments Were Conducted and Which Students Were 
Tested? 

The mathematics and science literacy test was designed to measure the mathematics 
and science learning of all final-year students who are at the point of leaving school 
and entering the workforce or postsecondary education, regardless of their school 
curriculum. These students may have specialized in mathematics and science in 
secondary school or have concentrated their studies in other areas, depending on the 
curricula offered in the participating countries. The mathematics and science literacy 
study is designed to provide information about how prepared the overall population 
of school leavers in each country is to apply knowledge in mathematics and science 
to meet the challenges of life beyond school. 

The advanced mathematics test was designed to measure learning of advanced 
mathematics concepts among final-year students who have studied advanced math- 
ematics. These students are at the point of leaving secondary school, and many will 
go on to further education in university or to another form of postsecondary education. 
Many of the mathematicians, scientists, engineers, medical practitioners, and business 
leaders of the future will be drawn from this group. In all countries that participated 
in the advanced mathematics assessment, the subpopulation of students tested had 
taken courses in advanced mathematics and was in the final year of secondary school 
at the time of testing. The exact definition of the subpopulation tested, however. 



5 Beaton, A.E., Martin, M.O., Mullis, I.V.S., Gonzalez, E.J., Smith, T.A., and Kelly, D.L. 1 1996). Science 
Achievement in the Middle School Years: IEA 's Third International Mathematics and Science Study (TIMSS). 
Chestnut Hill, MA: Boston College. 

6 Harmon, M., Smith, T.A., Martin, M.O., Kelly, D.L., Beaton, A.E., Mullis, I.V.S., Gonzalez, E.J., and Orpwood, 
G. 1 1 997). Performance Assessment in IEA s Third International Mathematics and Science Study (TIMSS). 



Chestnut Hill, MA: Boston College. 
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varied across countries in terms of which courses and how much advanced mathematics 
the students had taken (see Appendix A for more details). In addition to reporting 
achievement in advanced mathematics overall, this report presents achievement in 
three advanced mathematics content areas: numbers and equations; calculus; 
and geometry. 

The physics test was designed to measure learning of physics concepts and knowledge 
among final-year students who have studied physics. These students too are about 
to leave secondary school, and many will go on to university or other postsecondary 
education. The physics study was designed to provide information about how prepared 
the population of school leavers that has taken physics is to pursue higher education 
or occupations in science. In all countries the students participating in the physics 
testing had taken physics and were in the final year of secondary school at the time 
of testing, but the exact definition of the population varied across countries in terms 
of which courses and how much physics the students had taken (see Appendix A for 
more details). In addition to reporting achievement in physics overall, this report 
presents achievement in five physics content areas: mechanics; electricity and 
magnetism; heat; wave phenomena; and modem physics - particle physics, quantum 
and astrophysics, and relativity. 



Which Countries Participated? 

Table 1 shows the countries that participated in the assessment of students in their 
final year of secondary school in mathematics and science literacy, advanced math- 
ematics, and physics. Each participating country designated a national center to 
conduct the activities of the study and a National Research Coordinator (NRC) to 
assume responsibility for the successful completion of these tasks. 7 For the sake of 
comparability, all testing was conducted at the end of the school year. Most countries 
tested the mathematics and science achievement of their students at the end of the 
1994-95 school year, most often in May and June of 1995. The three countries on 
a Southern Hemisphere school schedule (Australia, New Zealand, and South Africa) 
tested from August to December 1995, which was late in the school year in the 
Southern Hemisphere. Students in Australia were tested in September to October; 
students in New Zealand were tested in August; and students in South Africa were 
tested in August to December 1995. Three countries tested their final-year students 
(or a subset of them) at the end of the 1995-96 school year. Iceland tested its final- 
year students in 1996; Germany tested its gymnasium students in 1996; and Lithuania 
tested the students in vocational schools in 1996. In Germany and Lithuania, all 
other students included in the TIMSS assessment were tested in 1995. 
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7 Appendix F lists the National Research Coordinators as well as the members of the TIMSS advisory 
committees. 
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Countries Participating in Testing of Students in Their Final Year of Secondary School* 



Mathematics 
and Science 
Literacy 



® Australia 
® Austria 
® Canada 

* Cyprus 

® Czech Republic 
® Denmark 
® France 
9 Germany 
® Hungary 
® Iceland 
® Israel 1 
3 Italy 
® Lithuania 
® Netherlands 
® New Zealand 
® Norway 

® Russian Federation 

* Slovenia 

9 South Africa 
® Sweden 

* Switzerland 

® United States 



Advanced 

Mathematics 



® Australia 
9 Austria 

• Canada 
® Cyprus 

® Czech Republic 
® Denmark 

• France 

® Germany 
® Greece 
8 Israel 1 
0 Italy 

• Lithuania 

® Russian Federation 

• Slovenia 
® Sweden 

® Switzerland 
® United States 



Physics 



• Australia 
® Austria 

• Canada 

• Cyprus 

® Czech Republic 

• Denmark 

• France 

• Germany 
® Greece 

9 Israel 1 
® Italy 2 
® Latvia 
® Norway 

• Russian Federation 
® Slovenia 

® Sweden 
® Switzerland 
9 United States 



* See Appendix A for characteristics of students tested. 

1 Because the characteristics of its sample are not completely known, achievement results for Israel are provided in Appendix D. 

2 Because it had a small sample for the physics testing, Italy’s physics achievement results are provided in Appendix D. 
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What Are the Differences in Upper Secondary Education Systems? 

The countries participating in TIMSS vary greatly with respect to their upper secondary 
education systems. Some countries provide comprehensive education to students in 
their final years of school, while in other countries students might attend academic, 
vocational, or technical schools. Some countries fall between these extremes, their 
students being enrolled in academic, vocational, technical, or general programs of 
study within the same schools. Across countries the definitions of academic, vocational, 
and technical programs also vary, as do the kinds of education and training students 
in these programs receive. 

There also are variations across and within countries with respect to the grades 
representing the final year of schooling. In some countries, all students in their final 
year of schooling are in the same grade (e.g., secondary schooling ends for all students 
in grade 12). In other countries, determining the final year of schooling is much more 
complicated because there are one or more academic tracks, one or more vocational 
tracks, and apprenticeship programs. In these countries, the final year of schooling 
may vary by track, with some students completing secondary school after a two-, three-, 
or four-year upper secondary program, depending on the type of school or program 
of study. Furthermore, determining when schooling in vocational programs is completed 
is not always straightforward. 

The differences across countries in how education systems are organized, how students 
proceed through the upper secondary system, and when students leave school posed 
a challenge in defining the target populations to be tested in each country and interpret- 
ing the results. In order to make valid comparisons of students’ performance across 
countries, it is critical that there be an understanding of which students were tested 
in each country, that is, how each country defined the target population. It also is 
important to know how each upper secondary education system is structured and how 
the tested students fit into the system as a whole. In order to provide a context for 
interpreting the achievement results presented in this report, Appendix A summarizes 
the structure of the upper secondary system for each country, and specifies the grades 
and tracks (programs of study) in which students were tested for TIMSS. 8 




Additional information about the education systems con be found in Rabitoille, D.F. (Ed.). (1997). National 
Contexts for Mathematics and Science Education: An Encyclopedia of the Education Systems Participating in 
TIMSS. Vancouver, B.C.: Pacific Educational Press. 
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TheTIMSS Coverage Index: What Percent of the School=Leaving 
Age Cohort Was Tested? 

Historically, an important difference between education systems was the proportion 
of an age cohort that successfully completed upper secondary education. In the 1960s, 
for example, completion rates among OECD countries ranged from more than 80% 
in the United States to between 17% and 33% in southern European countries. 9 One 
of the most significant developments in education systems around the world in the 
years since then has been the large increase in the number of students completing 
upper secondary education, with many countries catching up with the United States; 
yet there remains considerable variation among countries in completion rates. In order 
to avoid unwittingly comparing the elite students in one country with the more general 
population in another, therefore, it is important to be aware of the extent to which 
the upper secondary system in each country includes the total student population. 

So as to learn how much of the school-leaving age cohort was still in school and 
represented by the TIMSS sample, a TIMSS Coverage Index (TCI) was computed 
for each country. The TCI is an estimate of the percentage of the school-leaving age 
cohort covered by the TIMSS final-year student sample. It reflects any omissions 
from the sample, such as students who were excluded because of handicap or who 
had dropped out of school, and, in some countries, tracks or educational programs 
that were not covered by the TIMSS sample. The TCI was computed by forming a 
ratio of the size of the student population covered by the TIMSS sample, as estimated 
from the sample itself, to the size of the school-leaving age cohort, which was 
derived from official population census figures supplied by each country. 10 

Countries with high TCIs have most of their students still in school, and have covered 
this population with their TIMSS sample. Countries with low TCIs have fewer 
students still in school, or have excluded some components of their system from 
their sample (or both). Table 2 presents the TCI for each country, and also shows 
the two parts of the portion of the school-leaving age cohort not covered by the 
TIMSS sample: system components and students excluded by the country, and others - 
primarily young people who chose not to complete upper secondary education. The 
percentage of the age cohort covered by the TIMSS sample (the TCI), the percentage 
excluded from the sample, and the percentage of others not covered combine to form 
100% of the school-leaving age cohort. For example, Australia has a TCI of 68.1%, 
which indicates that the TIMSS sample of final-year students covers just over two- 
thirds of the school-leaving age cohort. Of the remainder, 4% have been excluded 
from the sample, and the remaining 27.9% are presumably no longer attending 
school. The TCI for Cyprus is lower (47.9%), partly because Cyprus excluded 
students in private schools and in vocational programs (13.5%), and partly because a 
greater percentage of the age cohort is no longer attending school (38.6%). 




9 OECD ( 1996). Education at a Glance ■ Analysis. Paris: Organization for Economic Cooperation and 
Development. 

10 For more information on the TIMSS Coverage Index, see Appendix B. 

33 



17 



INTRODUCTION 



Table 2 



TIMSS Coverage Indices (TCIs) 



Country 


TIMSS 
Coverage 
Index (TCI)* 


Sample 

Exclusions t 


Others Not 
Covered 


Notes on Exclusions 


Australia 


68.1% 


4.0% 


27.9% 




Austria 


75.9% 


16.8% 


7.3% 


Colleges and courses lasting less than 3 years excluded 


Canada 


70.3% 


6.8% 


22.9% 




Cyprus 


47.9% 


13.5% 


38.6% 


Private and vocational schools excluded 


Czech Republic 


77.6% 


5.0% 


17.4% 




Denmark 


57.7% 


1 .3% 


41.0% 




France 


83.9% 


0.9% 


15.3% 




Germany 


75.3% 


9.6% 


- 




1 Greece 


10.0% 


56.8% 


33.2% 


Only students having taken advanced mathematics and 
physics included 


Hungary 


65.3% 


0.1% 


34.6% 


Iceland 


54.5% 


0.0% 


45.4% 




2 Israel 


- 


- 


- 




Italy 


51 .5% 


0.5% 


48.0% 




1 Latvia 


3.0% 


16.8% 


80.3% 


Only students having taken physics included 


Lithuania 


42.5% 


0.0% 


57.5% 




Netherlands 


78.0% 


21 .5% 


0.5% 


Apprenticeship programs excluded 


New Zealand 


70.5% 


0.0% 


29.5% 




Norway 


84.0% 


3.3% 


12.7% 




Russian Federation 


48.1% 


36.3% 


15.7% 


Vocational schools and non-Russian speaking students 
excluded 


Slovenia 


87.8% 


5.6% 


6.6% 




South Africa 


48.9% 


0.0% 


51.1% 




Sweden 


70.6% 


0.2% 


29.2% 




Switzerland 


81 .9% 


2.1% 


16.0% 




United States 


63.1% 


2.5% 


34.5% 





SOURCE: IEA Third International Mathematics and Science Study (TIMSS), 1995-96. 



* TIMSS Coverage Index (TCI): Estimated percentage of school-leaving age cohort covered by TIMSS sample. See Appendix B for details. 

t Percentage different from that reported in Table B.4 because this is based on the entire school-leaving age cohort rather than the population 
of those students attending school. 

1 Results for Greece are reported only for advanced mathematics and physics; results for Latvia are available only for physics. 

2 The TCI could not be computed for Israel. 
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TIMSS Coverage Indices (TCIs) for Advanced Mathematics and Physics 



Country " > 


Percentage of 
Students in Sam p le 
Haying Taken 

Advanced 1 Mathematics 


Mathematics TIMSS 
Coverage Index 
(MTC!)* 


Percentage of 
Students in Sample 
Having Taken Physics 


Physics TIMSS 
Coverage Index (PTCI) 


Australia 


23.1% 


15.7% 


18.5% 


12.6% 


Austria 


43.9% 


33.3% 


43.5% 


33.1% 


Canada 


22.3% 


15.6% 


19.4% 


13.7% 


Cyprus 


18.5% 


8.8% 


18.5% 


8.8% 


Czech Republic 


14.1% 


11.0% 


14.1% 


11.0% 


Denmark 


35.7% 


20.6% 


5.5% 


3.2% 


France 


23.8% 


19.9% 


23.8% 


19.9% 


Germany 


34.9% 


26.3% 


11.2% 


8.4% 


1 Greece 


- 


10.0% 


- 


10.0% 


2 Israel 


- 


- 


- 


- 


Italy 


27.4% 


14.1% 


16.7% 


8.6% 


3 Latvia 


- 


- 


- 


3.0% 


Lithuania 


6.1% 


2.6% 


- 


- 


Norway 


- 


- 


10.0% 


8.4% 


Russian Federation 


4.2% 


2.0% 


3.2% 


1 .5% 


Slovenia 


85.9% 


75.4% 


43.9% 


38.6% 


Sweden 


23.0% 


16.2% 


23.1% 


16.3% 


Switzerland 


17.4% 


14.3% 


17.3% 


14.2% 


United States 


21.8% 


13.7% 


22.9% 


14.5% 



SOURCE: IEA Third International Mathematics and Science Study (TIMSS), 1995-96. 



* MTCI: Estimated percentage of school-leaving age cohort covered by TIMSS sample of advanced mathematics students. See Appendix A for 
characteristics of students sampled and Appendix B for details about the MTCI. 
t PTCI: Estimated percentage of school-leaving age cohort covered by TIMSS sample of physics students. See Appendix A for characteristics 
of students sampled and Appendix B for details about the PTCI. 

1 Greece only sampled students having taken advanced mathematics and physics. 

2 The MTCI and the PTCI could not be computed for Israel. 

3 Latvia only sampled students having taken physics. 

Note: Hungary, Iceland, the Netherlands, New Zealand, and South Africa did not participate in the advanced mathematics and physics testing. 
Norway did not participate in the advanced mathematics testing and Lithuania did not participate in the physics testing. 
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TIMSS also tested two overlapping subpopulations of the final-year student population: 
students having taken advanced mathematics, and students having taken physics. In 
most countries, each group consists of a minority of students from the final-year 
student population. Table 3 presents the percentage of students in the final-year 
sample having taken advanced mathematics and the percentage having taken physics. 
Apart from Slovenia, where a large percentage of upper secondary students take 
advanced mathematics, the percentage having taken advanced mathematics varies 
from about 4% in the Russian Federation to about 44% in Austria, with a similar 
range in physics. 

In order to quantify the coverage of the advanced mathematics and physics samples 
and help interpret the achievement results for these students, TIMSS computed a 
Mathematics TIMSS Coverage Index (MTCI) and a Physics TIMSS Coverage 
Index (PTCI), as shown in Table 3. The MTCI is the overall TCI multiplied by the 
percentage of the final-year sample having taken advanced mathematics. For example, 
in Australia 23.1% of the final-year sample had taken advanced mathematics. 
Multiplying this by the TCI (68.1%, from Table 2) gives a MTCI of 15.7%, as shown 
in the second column of Table 3. This implies that about 16% of the school-leaving 
age cohort in Australia had taken advanced mathematics in upper secondary school. 
Similarly, the PTCI for Australia is 12.6%, as shown in the fourth column of Table 3. 

How Does TIMSS Document Compliance with Sampling 
Guidelines? 

In addition to a clear definition of the populations assessed, valid samples and high 
participation rates in each country are crucial to the quality and success of any 
international comparative study. The accuracy of the survey results depends on the 
quality of sampling information and particularly on the quality of the samples. TIMSS 
developed procedures and guidelines to ensure that the national samples were of the 
highest quality possible. Standards for coverage of the target population and 
participation rates were established, as were clearly documented procedures on how 
to obtain the national samples. Despite efforts to meet the TIMSS specifications, 
some countries did not do so. These countries are specially footnoted or shown in 
separate sections of the tables in this report. 11 

Despite the differences in the structure of the upper secondary systems and the 
proportion of the school-leaving age cohort assessed, and the difficulties some 
countries had in meeting the TIMSS sampling requirements, the assessment of 
final-year students provides valuable comparative information about student 
achievement. This report describes in as much detail as possible which students 
were tested in each country, so that the achievement results can be understood 
and compared appropriately. 



11 The TIMSS sampling requirements and the outcomes of the sampling procedures are described in Appendix B. 
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How Do Country Characteristics Differ? 

International studies of student achievement provide useful information about 
student performance and instructional practices. The benefits of these studies, however, 
are accompanied by the problems of comparing achievement across countries, cultures, 
and languages. In TIMSS, extensive efforts were made to attend to these issues through 
careful planning and documentation, cooperation among the participating countries, 
standardized procedures, and rigorous attention to quality control throughout. 12 

Beyond the integrity of the study procedures, the results of comparative studies such 
as TIMSS also need to be considered in light of the larger contexts in which students 
are educated and the systemwide factors that might influence students’ opportunity 
to learn. A number of these factors are summarized in Appendix A and more fully 
described in National Contexts for Mathematics and Science Education: An Ency- 
clopedia of the Education Systems Participating in TIMSS.' 3 However, differences 
among the participating countries go beyond how their educational systems are 
organized. Selected demographic characteristics of the TIMSS countries are presented 
in Table 4, and Table 5 contains information about public expenditure on education. 
These tables show that some of the TIMSS countries are densely populated and others 
are more rural, some are large and some small, and some expend considerably more 
resources on education than others. Although these factors do not necessarily 
determine high or low performance in mathematics or the sciences, they do provide a 
context for considering the difficulty of the educational task from country to country. 

Describing students’ educational opportunities also requires an understanding of the 
knowledge and skills that students are supposed to master. To help complete the picture 
of educational practices in the TIMSS countries, mathematics and curriculum 
specialists in each country provided detailed categorizations of their curriculum guides, 
textbooks, and curricular materials. The initial results from this effort can be found in 
two reports, entitled Many Visions, Many Aims: A Cross -National Investigation of 
Curricular Intentions in School Mathematics, and Many Visions, Many Aims: A 
Cross-National Investigation of Curricular Intentions in School Science.' 4 

Depending on the education system, students’ learning goals are commonly set at 
one of three levels: the national or regional level, the school level, or the classroom 
level. Some countries are highly centralized, with the ministry of education (or highest 
authority in the system) having exclusive responsibility for making the major decisions 
governing the direction of education. In others, such decisions are made regionally 
or locally. Each approach has its strengths and weaknesses. Centralized decision- 
making can add coherence in curriculum coverage, but may constrain a school or 
teacher’s flexibility in tailoring instruction to the needs of students. 
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12 Appendix B summarizes the procedures used and cites references to TIMSS methodology. 

13 Robitaille, D.F. (Ed.). ( 1997). Notional Contexts for Mathematics and Science Education: An Encyclopedia of 
the Education Systems Participating in TIMSS. Vancouver, B.C.: Pacific Educational Press. 

14 Schmidt, W.H., McKnight, C.C., Valverde, G. A., Houang, R.T., and Wiley, D. E. (1997). Many Visions, 

Many Aims: A Cross-National Investigation of Curricular Intentions in School Mathematics. Dordrecht, the 
Netherlands: Kluwer Academic Publishers. Schmidt, W.H., Raizen, S.A., Britton, E.D., Bianchi, L.J., and Wolfe, 
R.G. (1997). Many Visions, Many Aims: A Cross-National Investigation of Curricular Intentions in School 
Science. Dordrecht, the Netherlands: Kluwer Academic Publishers. 
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Table 4 



Selected Demographic Characteristics of TIMSS Countries 



Country 


Population 
Size (1,000)' 


Area of Country 
(1000 Square 
Kilometers) 2 


Density 

(Population per 
Square 
Kilometer)* 


Percentage of 
Population 
Living in Urban 
Areas 


Life 

Expectancy 4 


Percent in 
Secondary 
School 5 


Australia 


17843 


7713 


2.29 


84.8 


77 


84 


Austria 


8028 


84 


95.28 


55.5 


77 


107 


Canada 


29248 


9976 


2.90 


76.7 


78 


88 


Cyprus 


726 


9 


77.62 


53.6 


77 


95 


Czech Republic 


10333 


79 


130.99 


65.3 


73 


86 


Denmark 


5205 


43 


120.42 


85.1 


75 


114 


France 


57928 


552 


104.56 


72.8 


78 


106 


Germany 


81516 


357 


227.39 


86.3 


76 


101 


Greece 


10426 


132 


78.63 


64.7 


78 


99 


Hungary 


10261 


93 


110.03 


64.2 


70 


81 


Iceland 


266 


103 


2.56 


91.4 


79 


103 


Israel 


5383 


21 


252.14 


90.5 


77 


87 


Italy 


57120 


301 


189.36 


66.6 


78 


81 


Latvia 


2547 


65 


40.09 


72.6 


68 


87 


Lithuania 


3721 


65 


57.21 


71.4 


69 


78 


Netherlands 


15381 


37 


409.30 


88.9 


78 


93 


New Zealand 


3493 


271 


12.78 


85.8 


76 


104 


Norway 


4337 


324 


13.31 


73.0 


78 


116 


Russian Federation 


148350 


17075 


8.70 


73.2 


64 


88 


Slovenia 


1989 


20 


97.14 


62.7 


74 


85 


South Africa 


40539 


1221 


32.46 


50.5 


64 


77 


Sweden 


8781 


450 


19.38 


83.1 


78 


99 


Switzerland 


6994 


41 


168.03 


60.6 


78 


91 


United States 


260650 


9809 


27.56 


76.0 


77 


97 



SOURCE: The World Bank, Social Indicators of Development, 1996. 



1 Estimates for 1994 based, in most cases, on a de facto definition. Refugees not permanently settled in the country of asylum are generally 
considered to be part of their country of origin. 

2 Area is the total surface area in square kilometers, comprising all land area and inland waters. 

3 Density is population per square kilometer of total surface area. 

4 Number of years a newborn infant would live if prevailing patterns of mortality at its birth were to stay the same throughout its life. 

5 Enrollment of students of all ages in the secondary school system as a percentage of the number of persons in the age group that attends 
secondary school. The age range varies across countries, but is usually 12-17. The percentage may be in excess of 100% if some pupils are 
younger or older than the country's standard range of secondary school age. 
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Table 5 



Public Expenditure on Education at Primary and Secondary Levels 1 in TIMSS Countries 



Country 


Gross National 
Product per Capita 
(US Dollars;) 2 


Gross National 
Product-pfer Capita 
(Inti. Dpilars) 3 


Public Expenditure on 
Education (Levels 1 & 2) 
as % of Gross 
National Product* 


Public Expenditure on. 
Education 
(Inti. Dollars per 
Capita) 6 * • 


Australia 


17980 


19000 


3.69 


701 


Austria 


24950 


20230 


4.24 


858 


Canada 


19570 


21230 


4.62 


981 


6 Cyprus 


10380 


- 


3.60 


- 


Czech Republic 


3210 


7910 


3.75 


297 


Denmark 


28110 


20800 


4.80 


998 


France 


23470 


19820 


3.61 


716 


Germany 


25580 


19890 


2.43 


483 


Greece 


7710 


11400 


2.27 


259 


Hungary 


3840 


6310 


4.31 


272 


Iceland 


24590 


18900 


4.77 


902 


Israel 


14410 


15690 


3.72 


584 


Italy 


19270 


18610 


2.89 


538 


Latvia 


2290 


5170 


2.85 


147 


Lithuania 


1350 


3240 


2.18 


71 


Netherlands 


21970 


18080 


3.30 


597 


New Zealand 


13190 


16780 


3.15 


529 


Norway 


26480 


21120 


5.26 


1111 


Russian Federation 


2650 


5260 


- 


- 


Slovenia 


7140 


- 


4.20 


- 


South Africa 


3010 


- 


5.12 


- 


Sweden 


23630 


17850 


4.92 


878 


Switzerland 


37180 


24390 


3.72 


907 


United States 


25860 


25860 


4.02 


1040 



SOURCE: The World Bank Atlas, 1996; and UNESCO Statistical Yearbook, 1995 



1 The levels of education are based on the International Standard Classification of Education. The duration of Primary (level 1) and Secondary 
(level 2) vary depending on the country. 

2 Estimates for 1994 at current market prices in U.S. dollars, calculated by the conversion method used for the World Bank Atlas. 

(Source: The World Bank Atlas, 1996). 

3 Converted at purchasing power parity (PPP). PPP is defined as number of units of a country’s currency required to buy the same amounts of 
goods and services in the domestic market as one dollar would buy in the United States. (Source: The World Bank Atlas, 1996). 

4 Calculated by multiplying the Public Expenditure on Education as a % of GNP by the percentage of public education expenditure on the first 
and second levels of education. Figures represent the most recent figures released. (Source: UNESCO Statistical Yearbook, 1995). 

5 Calculated by multiplying the GNP per Capita (Inti. Dollars) column by Public Expenditure on Education. 

6 GNP per capita figure for Cyprus is for 1993. 

(-) A dash indicates the data were unavailable. 
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Figures 1, 2, and 3 show the degree of centralization in the TTMSS countries regarding 
decision-making about curriculum syllabi, textbooks, and examinations. Fourteen 
of the TIMSS participants reported nationally centralized decision-making about 
curriculum. Fewer countries reported nationally centralized decision-making about 
textbooks: six participants were in this category. Eight countries reported nationally 
centralized decision-making about examinations. Regional decision-making about 
these three aspects of education does not appear to be very common, with only a few 
countries reporting it for curriculum syllabi and textbooks, and none reporting it for 
examinations. 

Most countries reported having centralized decision-making for one or two of the 
areas and “not centralized” decision-making for one or two of the areas. Two countries, 
Lithuania and Norway, reported nationally centralized decision-making for all three 
areas: curriculum syllabi, textbooks, and examinations. Five countries - Australia, 
Hungary, Iceland, Latvia, and the United States - reported that decision-making is 
not centralized for any of these areas. 
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Centralization of Decision-Making Regarding Curriculum Syllabi 






Countries are in the “Nationally Centralized” category regarding curriculum if the highest level of 
decision-making authority within the educational system (e.g., the ministry of education) has exclusive 
responsibility for or gives final approval of the syllabi for courses of study. If curriculum syllabi are 
determined at the regional level (e.g., state, province, territory), a country is in the “Regionally 
Centralized” category. If syllabi for courses of study are not determined nationally or regionally, a 
country is in the “Not Centralized” category. 




Nationally 

Centralized 



Austria 

Cyprus 

Czech Republic 
Denmark 
France 
Greece 
Israel 
Italy 

Lithuania 
New Zealand 
Norway 1 
Slovenia 
South Africa 
Sweden 2 



Canada 

Germany 

Switzerland 3 



Not 

Centralized 



Australia 4 

Hungary 5 

Iceland 

Latvia 

Netherlands 6 
Russian Federation 
United States 



1 Norway: The National Agency of Education provides goals which schools are required to work towards. Schools have the freedom to 
implement the goals based on local concerns. 

2 Sweden: The National Agency of Education provides goals which schools are required to work towards. Schools have the freedom to 
implement the goals based on local concerns. 

3 Switzerland: Decision-making regarding curricula in upper secondary varies across the cantons and the types of education. 

4 Australia: Students tested in TIMSS were educated under a decentralized system. Reforms beginning in 1994 are introducing regionally 
centralized (state-determined) curriculum guidelines. 

5 Hungary: Hungary is in the midst of changing from a highly centralized system to one in which local authorities and schools have more 
autonomy. 

6 Netherlands: The Ministry of Education sets core objectives (for subjects in primary education and in ‘basic education 1 at lower secondary 
level) and goals/objectives (for subjects in the four student ability tracks in secondary education) which schools are required to work towards. 

Schools have the freedom, though, to decide how to reach these objectives. 

SOURCE: IEA Third International Mathematics and Science Study (TIMSS), 1995. Information provided by TIMSS National Research Coordinators. 
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INTRODUCTION 




Centralization of Decision-Making Regarding Textbooks 



Criteria 



Countries are in the "Nationally Centralized” category regarding textbooks if the highest level of 
decision-making authority within the educational system (e.g., the ministry of education) has exclusive 
responsibility for determining the approved textbooks. If textbooks are selected from a regionally 
approved list (e.g., state, province, territory), a country is in the "Regionally Centralized" category. If 
that decision-making body has less than exclusive responsibility for determining the approved 
textbooks, a country is in the "Not Centralized" category. 



Nationally 

Centralized 




Regionally 

Centralized 


Austria 




Canada 


Cyprus 




Germany 


Greece 




South Africa 


Lithuania 

Norway 

Slovenia 




Switzerland 1 




Australia 
Czech Republic 
Denmark 
France 
Hungary 2 
Iceland 
Israel 
Italy 
Latvia 

Netherlands 
New Zealand 
Russian Federation 
Sweden 
United States 



1 Switzerland: Decision-making regarding textbooks in upper secondary varies across the cantons and the types of education. 

2 Hungary: Hungary is in the midst of changing from a highly centralized system to one in which local authorities and schools have more autonomy. 

SOURCE: IEA Third International Mathematics and Science Study (TIMSS), 1995-96. Information provided byTIMSS National Research Coordinators. 
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INTRODUCTION 




Centralization of Decision-Making Regarding Examinations 






Countries are in the "Nationally Centralized" category regarding examinations if the highest level of 
decision-making authority within the educational system (e.g., the ministry of education) has exclusive 
responsibility for or gives final approval of the content of examinations. The notes explain during which school 
years the examinations are administered. If that decision-making body has less than 
exclusive responsibility for or final approval of the examination content, the country is in the "Not 
Centralized" category. 



Nationally 

Centralized 



Denmark 1 
Italy 2 
Lithuania 
Netherlands 3 
New Zealand 4 
Norway 5 

Russian Federation 6 
South Africa 



Not 

Centralized 



Australia 7 

Austria 

Canada 

Cyprus 

Czech Republic 
France 
Germany 8 
Greece 
Hungary 
Iceland 
Israel 9 
Latvia 10 
Slovenia 11 
Sweden 12 
Switzerland 
United States 



1 Denmark: Written examinations are set and marked centrally. The Ministry of Education sets the rules and framework for oral examinations. 
However, oral examinations are conducted by the pupil's own teacher, together with a teacher from another school as an external 
(ministry-appointed) examiner. 

2 Italy: At the end of senior secondary courses lasting four or more years, students who have positive evaluations write the final examination, the 
esame di maturity. Written papers are determined by the Ministry of Education. 

3 Netherlands: School-leaving examinations consisting of a centralized part and a school-bound part are taken in the final grades of the four student 
ability tracks in secondary education. 

4 New Zealand: Centralized examinations taken at Years 11,12, and 13. Centralized national monitoring at Years 4 and 8. 

5 Norway: Written examinations are set and marked centrally. The Ministry of Education sets the rules and framework for oral examinations. However, 
oral examinations are conducted by the pupil's own teacher, together with a teacher from another local school or an external (ministry-appointed) 
examiner. 

6 Russian Federation: Centralized examinations are taken in Grades 9 and 11 in mathematics and Russian/literature. 

7 Australia: Not centralized as a country, but low-stakes statewide population assessments are undertaken in most states at one or more of Grades 
3, 5, 7, and 10. In most states, centralized examinations are taken at Grade 12. 

8 Germany: Not centralized as a country, but is centralized within 6 (of 16) federal states. 

9 Israel: Centralized examinations are taken at the end of secondary school that affect opportunities for further education. 

10 Latvia: Centralized examinations can be taken at Grade 9 and Grade 12. 

11 Slovenia: Two-subject national examinations are taken after Grade 8 (end of compulsory education); five-subject externally-assessed baccalaureat 
after Grade 12 for everyone entering university. 

12 Sweden: There are no examinations in Sweden. 

SOURCE: IEA Third International Mathematics and Science Study (TIMSS), 1995-96. Information provided by TIMSS National Research Coordinators. 
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— Chapter 1 



International Student Achievement in 
Mathematics and Science Literacy 



This chapter summarizes achievement on the TIMSS mathematics and science literacy 
test for each of the participating countries. The test was designed to measure the 
mathematics and science learning of all final-year students, regardless of their school 
curriculum. These students, who are at the point of leaving school and entering the 
workforce or postsecondary education, may have specialized in mathematics and 
science in upper secondary school or have concentrated their studies in other areas. 
The mathematics and science literacy study is intended to provide information about 
how prepared all the school leavers in each country are to apply their knowledge 
in mathematics and science to meet the challenges of life beyond school. 

Comparisons are provided for the populations of school leavers tested in each of the 
countries. The relationship between achievement and the population tested is examined 
from several perspectives, because not all of the countries were able to provide 
coverage of the entire school-leaving age cohort. In all of the participating countries, 
some members of the school-leaving age cohort no longer attended school, having 
completed their compulsory education or having dropped out for a variety of reasons. 
In some of the countries, portions of the students still attending school were not 
tested, usually because they were in on-site vocational education situations and 
difficult to locate for the testing. 

How Does Performance Compare for the Students Participating 
in the Testing? 

Table 1.1 presents the mean (or average) achievement for the 21 countries that 
participated in the mathematics and literacy study for students in their final year of 
secondary school. 1 The mean for each country can be compared with the international 
average of 500, which represents the average across the means for each of the 21 
participants shown in the table. A number of countries had mean achievement well 
above the international average of 500, and others well below that level. A triangle 
pointing up next to the mean indicates that the country’s performance was signifi- 
cantly higher than the international average, while a triangle pointing down indicates 
that its performance was significantly lower. Among the countries meeting the 
TIMSS sampling guidelines, Sweden, Switzerland, and New Zealand performed 
above the international average. 




1 TIMSS used item response theory (IRT) to summarize the achievement for mathematics literacy and for science 
literacy on two separate scales, each with a mean of 500 and a standard deviation of 100. Scaling averages 
students' responses to the subsets of items they took in a way that accounts for differences in the difficulty of 
those items. It allows students' performance to be summarized on a common metric even though individual 
students responded to different items in the mathematics and science literacy tests. The composite results for 
mathematics and science literacy represent an average of the results on the mathematics and science literacy 
scales (see Chapter 2 for separate results for mathematics and science literacy). For more detailed 
information, see the "IRT Scaling and Data Analysis" section of Appendix B. 
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Table 1.1 



Distributions of Mathematics and Science Literacy Achievement for Students in Their 
Final Year of Secondary School* * 



Mathematics and Science Literacy 
Achievement Scale Score 



Country 


Mean 


TCI * 


Average 

Age 


Sweden 


▲ 555(4.3) 


71% 


18.9 


Switzerland 


▲ 531 (5.4) 


82% 


19.8 


New Zealand 


▲ 525(4.7) 


70% 


17.6 


Hungary 


▼ 477(3.0) 


65% 


17.5 


Russian Federation 


T 476(5.8) 


48% 


16.9 


Czech Republic 


• 476(10.5) 


78% 


17.8 


Lithuania 


▼ 465(5.8) 


43% 


18.1 


Cyprus 


T 447(2.5) 


48% 


17.7 



□= 



Countries Not Satisfying Guidelines for Sample Participation Rates (See Appendix B for Details) 



Australia 


• 


525 (9.5) 


68% 


17.7 


2 Austria 


▲ 


519(5.4) 


76% 


19.1 


Canada 


▲ 


526 (2.6) 


70% 


18.6 


France 


• 


505 (4.9) 


84% 


18.8 


Iceland 


▲ 


541 (1.6) 


55% 


21.2 


1 Italy 


▼ 


475 (5.3) 


52% 


18.7 


Norway 


▲ 


536 (4.0) 


84% 


19.5 


United States 


▼ 


471 (3.1) 


63% 


18.1 



T 



n r 






T 



□ 



Countries With Unapproved Student Sampling (See Appendix B for Details) 



Germany 



496 (5.4) 



75% 



19.5 



| Countries With Unapproved Sampling Procedures and Low Participation Rates (See Appendix B for Details ) 



Denmark 


▲ 


528 (3.2) 


58% 


19.1 


2 Netherlands 


▲ 


559 (4.9) 


78% 


18.5 


Slovenia 


• 


514(8.2) 


88% 


18.8 


South Africa 


T 


352 (9.3) 


49% 


20.1 



EE 



□ 



200 250 300 350 400 450 500 550 600 650 700 750 800 



I Percentiles of Performance — | 

5th 25th 75th 95th 



International Average = 500 
(Average of All Country Means) 



Mean and Confidence Interval (±2SE) 



A = Country mean significantly higher than international mean 
▼ = Country mean significantly lower than international mean 

• = No statistically significant difference between country mean and international mean 



* See Appendix A for characteristics of students sampled. 

♦The TIMSS Coverage Index (TCI) is an estimate of the percentage of the school-leaving age cohort covered by the TIMSS final-year student 
sample (see Appendix B for more information). 

f Met guidelines for sample participation rates only after replacement schools were included (see Appendix B for details). 

1 National Desired Population does not cover all of International Desired Population (see Table B.4). 

2 National Defined Population covers less than 90 percent of National Desired Population (see Table B.4). 

( ) Standard errors appear in parentheses. Because results are rounded to the nearest whole number, some totals may appear inconsistent. 




SOURCE: IEA Third International Mathematics and Science Study (TIMSS), 1995-96. 
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Multiple Comparisons of Mathematics and Science Literacy Achievement for 
Students in Their Final Year of Secondary School* 

Instructions: Read across the row for a country to compare performance with the countries listed in the heading of the chart. The 
symbols indicate whether the mean achievement of the country in the row is significantly lower than that of the comparison country, 
significantly higher than that of the comparison country, or if there is no statistically significant difference between the two countries.* 



Country 


Netherlands 


Sweden 


Iceland 


1 Norway 


Switzerland 


Denmark 


Canada 


New Zealand 


Australia 


Austria 


Slovenia 


France 


Germany 


Hungary 


Czech Republic 


Russian Federation 


■S 


United States 


Lithuania 


Cyprus 


South Africa 


Netherlands 




• 


A 


A 


▲ 


A 


A 


▲ 


A 


A 


A 


A 


A 


▲ l 

| 


▲ 


▲ 


▲ 


A 


▲ 


A 


A 


Sweden 


• 




T 


T| 


•H 

▲ 


A 


1 

: ± 1 


▲ 


• 


A 


A 


A 


A 


▲ 


▲ 


▲ 


▲ 


A 


▲ 


A 


A 


Iceland 


▼ 


▼ 




• 


• 


A 


▲ 


▲ 


• 


A 


A 


A 


A 


▲ 


▲ 


}’ ▲ 


▲ 


A 


▲ 


A 


A 


Norway 


▼ 


▼ 


• 


















A 


A 


▲ 


▲ 


▲ 


▲ 


A 


▲ 


A 


A 


Switzerland 


▼ 


▼ . 


• 


• 


















A 


▲ 


▲ 


! ▲ 


▲ 


A 


▲ 


A 


A 


Denmark 


▼ : 


▼ . 


▼ 


• 


• 




• 


• 


• 


• 


• 


A 


A 


▲ 


▲ 


▲ 


▲ 


A 


A 


A 


A 


Canada 


▼ 


T 


▼ 


• 


• 


m 




• 


• 


• 


• 


A 


A 


▲ 


▲ 


▲ 


▲ 


A 


"a] 


1 

: i 


X 


New Zealand 


▼ : 


▼ 


▼ 


• 


• 


m 


• 




• 


• 


• 


• 


A 


▲ 


▲ 


▲ 


■Aj 


\~A 
| 


A 


A i 


1 A 


Australia 






























▲ 


A 


▲ 


i A 


A 


▲ 


▲ 


Austria 


▼ 


T 


















• 


• 


A 


▲ 


▲ 


"1 


T 


A 


A 


X 


▲ 


Slovenia 


▼ 


▼ 


▼ 


























▲ 


▲ 


A 


A 


! A 

1 


▲ 


France 


▼ 


▼ 


▼ 


▼ 


▼ ; 


▼ 


▼ 


• 


• 


• 


• 




• 


▲ 


• 


▲ 


▲ 


A 


A 


r 

i ▲ 

L 


▲ 


Germany 


▼ 


▼ 


▼ 


▼ 


▼ 


▼ 1 


▼ 


▼ 


• 


; ▼ 


• 


• 




▲ 


• 


• 


• 


A 


A 


i A 


▲ 


Hungary 


▼ 


▼ 


▼ 


▼ 


▼ 


▼ 


▼ 


▼ 


T 


▼ 


T 


▼ 


▼ 
















▲ 


Czech Republic 


▼ 


▼ 


T 


▼ 


▼ 


▼ 


▼ 


▼ 


























▲ 


Russian Federation 


▼ 


T 


▼ 


T 


▼ 


▼ 


▼ 


▼ 


▼ 


T 


T 


T 


• 


• 


• 




• 


• 


• 


▲ 


▲ 


Italy 


▼ 


▼ 


▼ 


▼ 


▼ 


▼ 


▼ 


▼ 


▼ 


T 


▼ 


▼ 


• 


• 


• 


• 




• 


• 


▲ 


▲ 


United States 


▼ 


▼ 


▼ 


▼ 


▼ 


▼ 


▼ 


▼ 


▼ 


T 


T 


▼ 


▼ 


• 


• 


• 


• 




• 


▲ 


▲ 


Lithuania 


▼ 


▼ 


T 


T 


▼ 


▼ 


T 


▼ 


▼ 


▼ 


▼ 


▼ 
















• 


▲ 


Cyprus 


▼ 


▼ 


▼ 


▼ 


▼ 


▼ 


▼ 


▼ 


▼ 


▼ 


▼ 


▼ 


▼ 


T 


• 


▼ 


▼ 


▼ 


• 




▲ 


South Africa 


▼ 


▼ 


▼ 


T 


▼ 


▼ 


▼ 


▼ 


▼ 


▼ 


▼ 


▼ 


▼ 


T 


▼ 


▼ 


▼ 


▼ 


T 


▼ 





Countries are ordered by mean achievement across the heading and down the rows. 



H Mean achievement 

significantly higher than 
comparison country 




No statistically significant 
difference from comparison 
country 



H Mean achievement 

significantly lower than 
comparison country 



* See Appendix A for characteristics of the students sampled. 

* Statistically significant at .05 level, adjusted for multiple comparisons. 

Countries shown in italics did not satisfy one or more guidelines for sample participation rates or student sampling (see Figure B.4). 
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The eight countries shown in decreasing order of mean achievement in the upper part 
of the table were judged to have met the TIMSS requirements for testing a representa- 
tive sample of the students in their nationally defined target populations. Lithuania 
is footnoted because its nationally defined population did not include part of the interna- 
tionally desired population, that is, it included only students in schools providing 
instruction in Lithuanian (see Table B.4). The Russian Federation and Cyprus are 
footnoted for not testing final-year students in some vocational tracks (see Table B.4). 
New Zealand is annotated because it met the sampling guidelines only after including 
replacement schools (see Table B.10). 

Although countries tried very hard to meet the TIMSS sampling requirements, many 
of them encountered resistance from schools, teachers, and students, and thus did 
not have the participation rates - 85% or higher for schools and for students both, 
or a combined rate of 75% - specified in the TIMSS guidelines. Obtaining a high 
participation rate for secondary school students is particularly challenging when 
participation is voluntary, because these students have many demands on their time. 
Also, their educational situations may make testing difficult; for example, in some 
countries students are engaged in on-site vocational training. The eight countries 
shown in the second category in Table 1.1 followed procedures but were unable to 
meet the TIMSS guidelines for sample participation. Beyond the difficulty of encour- 
aging students to attend the testing sessions, the five countries in the remaining two 
categories encountered various obstacles in implementing the prescribed methods 
for sampling schools or students within schools, usually because of the organization 
of the education system. Because Israel did not clearly document its procedures for 
sampling schools, its achievement results (unweighted) are presented in Appendix 
D. Appendix B includes a full discussion of the sampling procedures and outcomes 
for each country. 

As mentioned previously, some members of the school- leaving age cohort are no longer 
attending school. As explained in the Introduction, the degree of coverage of the 
entire school-leaving age cohort is indicated by the TIMSS Coverage Index (TCI). 

If the TCI also reflects exclusion of part of the final-year student population, that is 
noted for the countries concerned: the Russian Federation, Cyprus, Austria, and the 
Netherlands. (See Table 2 in the Introduction as well as Appendix B for more details 
about the TCI.) 

As shown in the table, there is quite a range in the TCI. About half the countries were 
able to cover 70% or more of the entire school-leaving age cohort by their in-school 
sampling procedures, including Slovenia (88%), France (84%), Norway (84%), 
Switzerland (82%), the Czech Republic (78%), the Netherlands (78%), Austria (76%), 
Germany (75%), Sweden (71%), New Zealand (70%), and Canada (70%). Countries 
covering less than half of this cohort included South Africa (49%), the Russian 
Federation (48%), Cyprus (48%), and Lithuania (43%). 

To aid in interpretation, the table also contains the average age of the students. 
Equivalence of chronological age does not necessarily mean that students have had 
the same number of years of formal schooling or have studied the same curriculum. 
Countries with a high percentage of older students may have policies that include 
retaining students in lower grades. Still, the average age, in combination with the 
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information about secondary school for each country presented in Appendix A, will 
provide an indication of the amount of schooling received by the students in 
each country. 

Table 1.1 also graphically shows the differences in average mathematics and science 
literacy achievement between the highest- and lowest-performing countries and the 
distribution of student performance within each country. Achievement for each 
country is shown for the 25th and 75th percentiles as well as for the 5th and 95th 
percentiles. 2 Each percentile point indicates the percentages of students performing 
below and above that point on the scale. For example, 25% of the students in each 
country performed below the 25th percentile for that country, and 75% performed 
above the 25th percentile. 

The range between the 25th and 75th percentiles represents performance by the middle 
half of the students. In contrast, performance at the 5th and 95th percentiles represents 
the extremes in lower and higher achievement. The dark boxes at the midpoints of 
the distributions are the 95% confidence intervals around the average achievement 
in each country. 3 

Comparisons can be made across the means and percentiles. For example, average 
performance in Sweden was comparable to or even exceeded performance at the 75th 
percentile in a number of countries, including Hungary, the Russian Federation, the 
Czech Republic, Lithuania, Cyprus, Italy, the United States, and especially South 
Africa. Also, the differences between the extremes in performance were very large 
in most countries. 

Figure 1.1 provides a method for making appropriate comparisons of overall mean 
achievement between countries. 4 The figure shows whether or not the differences 
in mean achievement between pairs of countries are statistically significant. Selecting 
a country of interest and reading across the table, a triangle pointing up indicates 
significantly higher performance than the country listed across the top, a dot indicates 
no significant difference, and a triangle pointing down indicates significantly lower 
performance. Countries shown in italics failed to satisfy one or more guidelines for 
sample participation rates or student sampling (see Appendix B for details). 

The Netherlands and Sweden, with mostly triangles pointing up, had significantly 
higher mean achievement than the other participating countries, and performed 
similarly. However, the Netherlands had particular difficulty in meeting the TIMSS 
sampling guidelines. Students in apprenticeship programs were excluded (21% of 
final-year students), and overall sample participation rates were very low (49%). 




2 Tables of the percentile values and standard deviations for all countries are presented in Appendix E. 

3 See the "IRT Scaling and Data Analysis" section of Appendix B for more details about calculating standard 
errors and confidence intervals. 

4 The significance tests in Figure 1.1 are based on a Bonferroni procedure for multiple comparisons that holds 
to 5% the probability of erroneously declaring the mean of one country to be different from another country. 
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Iceland, Norway, and Switzerland performed similarly, but had lower mean achieve- 
ment than the Netherlands and Sweden. However, of those three countries, only 
Switzerland met the sampling guidelines. It also can be observed that Switzerland 
and Norway had among the highest TCIs, 82% and 84%, respectively. Because the 
measurement in Australia was somewhat less precise than in many other participat- 
ing countries, it has a rather large confidence interval around its mean achievement, 
and tends to overlap with more countries than might otherwise be the case. Australia’s 
mean performance is more similar to that of Denmark, Canada, New Zealand, Austria, 
and Slovenia. Of these countries, only New Zealand met the sampling guidelines. 
France performed similarly to New Zealand, Australia, Austria, Slovenia, and Germany. 
Germany’s performance resembled that of Slovenia and France as well as the Czech 
Republic, the Russian Federation, and Italy. 

The lower-performing countries included Hungary, the Czech Republic, the Russian 
Federation, Italy, the United States, Lithuania, Cyprus, and South Africa. Only 
South Africa had significantly lower mean achievement than the other participating 
countries. Because of the pattern of relatively small differences from one country to 
the next, most countries had lower mean achievement than some countries, about the 
same mean achievement as some countries, and higher mean achievement than 
other countries. 



How Does Performance Compare, Taking Differences in 
Population Coverage into Account? 

Figure 1.2 shows the relationship between achievement and the TIMSS Coverage 
Index. It is designed to show whether countries may have achieved higher performance 
because they tested fewer students - in particular, a more elite group of students. In 
general, however, the relationship between performance and the degree of sample 
coverage of the entire school-leaving population shows that the higher-performing 
countries actually tended to have better coverage than the lower-performing countries. 5 
For example, the countries in the upper right comer of the graph had a high percentage 
of coverage of the entire school-leaving age cohort, as well as high performance. In 
particular, Switzerland exceeded 80% coverage, met the sampling guidelines, and 
performed above the international average. 

If anything, the countries with greater coverage (more than 70%) tended to have mean 
performance above the international average, and those with less coverage tended to 
perform below the international average. The only two high-performing countries 
with a low degree of coverage (less than 60%) were Denmark and Iceland. The 
remaining countries with coverage less than 60% all performed below the interna- 
tional average. 




5 The relationship between mathematics and science literacy achievement and the TIMSS Coverage Index has 
a carrelotian coefficient of 0.56. 
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Mean Mathematics and Science Literacy Achievement by TIMSS Coverage Index for Students 
in Their Final Year of Secondary School* 
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SOURCE: IEA Third International 
Mathematics and Science 
Study (TIMSS), 1995-96. 
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See Appendix A for characteristics of the students sampled. 

► The TIMSS Coverage Index (TCI) is an estimate of the percentage of the school-leaving age cohort covered by the TIMSS final-year student sample 
{see Appendix B for more information). 

Countries shown in italics did not satisfy one or more guidelines for sample participation rates or student sampling {see Figure B.4). 
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Table 1 .2 offers another way of examining performance, regardless of whether or 
not countries may have tested only their elite students. The table shows the 75th 
percentile of performance for the entire school-leaving age cohort for each country. 
It also presents the mean achievement of students performing above the 75th 
percentile - the top 25% of the students in the entire school-leaving age cohort for 
each country. The 75th percentile is the point on the mathematics and science literacy 
composite scale that divides the higher-performing 25% of the students from the 
lower- performing 75%. 

The 75th percentile is a useful summary statistic on which to compare performance 
across countries. It is used instead of the mean in this table because it can be reliably 
estimated even when scores from some members of the population are not available 
(that is, students in the school-leaving age cohort not included in the samples tested). 

As indicated by the TCI, the samples in some countries represented nearly all of the 
students in the school-leaving age cohort, while other countries covered only about 
half of these students. To compute the 75th percentile, TIMSS assumed that students 
in the school-leaving age cohort not covered by the sample in each country would 
score below the 75th percentile, primarily because they were no longer in the system 
by virtue of dropping out, being tracked out of the system, or being in difficult-to-test 
vocational tracks. The percentages of students assumed to be below the 75th percentile 
were added to the lower tail of the achievement distribution before calculating the 75th 
percentile using the modified distribution. 

Notwithstanding the additional difficulties in estimating achievement for the entire 
school-leaving age cohort for each country, rather than for the population of students 
actually tested, the results for the top 25% of the students in each country appear 
quite consistent with those obtained for the tested students. Of the countries meeting 
the sampling guidelines, Sweden, Switzerland, and New Zealand had the highest 
mean achievement for the top 25% of their school-leaving age cohorts. 

Figure 1.3 presents the country comparison chart for the top 25% of all students in 
the school-leaving age cohort. Among the top-performing countries, Sweden, the 
Netherlands, and Norway performed similarly, with Switzerland also performing 
similarly to Norway. In summary, the four top-performing countries had rankings 
very similar to those obtained for the populations of tested students. In particular, 
Sweden and Switzerland met the sampling guidelines and had high performance. 
Norway, too, performed very well even though participation rates were slightly 
below the guidelines (71%). The Netherlands also performed well, but had low 
participation rates (49%). 

Looking at the top 25% of performance for the school-leaving age cohort shows a 
block of countries with very similar mid-range performance, including New Zealand, 
Australia, Canada, Slovenia, Austria, Iceland, and Denmark. Germany, France, and 
the Czech Republic performed similarly but generally below the aforementioned 
countries. The lower-performing countries included Hungary and the United States, 
followed by Italy and the Russian Federation. Lithuania, Cyprus, and South Africa 
had lower performance than the other participating countries. The relative standing 
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Mathematics and Science Literacy Achievement for the Top 25 Percent® of All 
Students in the School-Leaving Age Cohort* 







TCI 



Sweden 


584 (6.3) 


654 (3.4) 


71% 


Switzerland 


575 (4.1) 


633 (2.6) 


82% 


f New Zealand 


559 (7.5) 


621 (1.9) 


70% 


Czech Republic 


508(12.0) 


584 (4.6) 


78% 


Hungary 


496 (2.8) 


563(3.1) 


65% 


2 Russian Federation 


464 (6.3) 


539 (4.8) 


48% 


1 Lithuania 


447 (6.8) 


519(3.6) 


43% 


2 Cyprus 


438 (4.0) 


501 (3.4) 


48% 


Countries Not Satisfying Guidefines for Sample Participation Rates (See Appendix B for Details) 


Australia 


555 (8.9) 


620 (4.8) 


68% 


2 Austria 


552 (5.6) 


610(4.2) 


76% 


Canada 


555 (5.6) 


613(2.6) 


70% 


France 


546 (8.0) 


592 (2.6) 


84% 


Iceland 


546 (3.0) 


609 (1 .4) 


55% 


1 Italy 


475 (5.6) 


543 (4.3) 


52% 


Nonway 


578 (3.9) 


641 (2.8) 


84% 


United States 


490 (3.1) 


559 (2.5) 


63% 


Countries With Unapproved Student Sampling (See Appendix B for Details) 


f Germany 


533 (5.6) 


593 (2.9) 


75% 


Countries With Unapproved Sampling Procedures and Low Participation Rates (See Appendix B for Details) 


Denmark 


539 (4.3) 


603 (2.3) 


58% 


2 Netherlands 


600 (6.0) 


653 (4.9) 


78% 


Slovenia 


560(9.6) 


612(4.9) 


88% 


South Africa 


328(4.4) 


412(11.4) 


49% 


International Average 


520(1.4) 


585 (0.9) 





SOURCE: IEA Third International Mathematics and Science Study (TIMSS), 1995*96. 



®To compute the 75th percentile, TIMSS assumed that the students in the school-leaving age cohort not tested would have scored below 
the 75th percentile and added them to the lower tail of the distribution. 

* See Appendix A for characteristics of students sampled. 

* Met guidelines for sample participation rates only after replacement schools were included (see Appendix B for details). 

1 National Desired Population does not cover all of International Desired Population (see Table B.4). 

2 National Defined Population covers less than 90 percent of National Desired Population (see Table B.4). 

( ) Standard errors appear in parentheses. Because results are rounded to the nearest whole number, some totals may appear inconsistent. 
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Figure 1.3 



Multiple Comparisons of Average Mathematics and Science Literacy Achievement 
for the Top 25 Percent of All Students in the School-Leaving Age Cohort* 

Instructions: Read across the row for a country to compare performance with the countries listed in the heading of the chart. The 
symbols indicate whether the mean achievement of the country in the row is significantly lower than that of the comparison country, 
significantly higher than that of the comparison country, or if there is no statistically significant difference between the two countries.* 
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Countries are ordered by mean achievement across the heading and down the rows. 



H Mean achievement 

significantly higher than 
comparison country 



H No statistically significant 
difference from comparison 
country 




Mean achievement 
significantly lower than 
comparison country 



* See Appendix A for characteristics of the students sampled. 

* Statistically significant at .05 level, adjusted for multiple comparisons. 

Countries shown in italics did not satisfy one or more guidelines for sample participation rates or student sampling (see Figure B.4). 
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of Iceland, Denmark, and the Russian Federation dropped somewhat in this analysis 
compared to the analysis based only on the samples of students tested. This may be 
because the assumptions of lower performance (below the 75th percentile) for students 
not represented in the sample do not completely apply in these two countries. For 
example, in the Russian Federation students not covered in the sampling included 
those in technical tracks that take mathematics and science, some of whom may 
have achieved above the 75th percentile. 



How Does Performance Compare by Gender? 

Table 1.3 shows the differences in mathematics and science literacy achievement by 
gender for the final-year students in each country. The table presents mean achievement 
separately for males and females for each country, as well as the difference between 
the means. The graphic representation of the gender difference, indicated by a bar, 
shows the amount of the difference, whether the direction of the difference favors 
females or males, and whether the difference is statistically significant (indicated 
by a darkened bar). As can be seen, all of the differences favored males rather than 
females, and all of the differences were statistically significant except in South Africa. 

Since the TIMSS science results for seventh and eighth grades showed significant 
gender differences favoring males to be pervasive across most countries, 6 and the 
direction of the differences in mathematics favored males much more often than 
females, 7 these results might have been anticipated for the secondary school students. 
Still, it is distressing to see such uniform gender differences favoring males in the 
general population of school-leaving students. There may be many reasons for such 
differences, including the fact that society encourages males more than females to 
have an interest in mathematics and science topics. This tends to lead to more outside 
activities in mathematics and science areas for males and taking more courses in 
these subjects, which serves to differentiate performance as students progress 
through school. 

Course-taking patterns are explored in more detail in Chapter 4 and in the second 
section of this report, which presents results for students having taken advanced 
mathematics courses (Chapter 5) and physics (Chapter 8) during their final years of 
secondary school. Briefly, however, while males take more mathematics and science 
courses than females in some countries, especially in physics, course-taking patterns 
alone do not seem to explain these pervasive gender differences for the overall 
population of school-leaving students. 



6 Beaton, A.E., Martin, M.O., Mullis, I.V.S., Gonzalez, E.J., Smith, T.A., and Kelly, D.L. (1996). Science Achievement 
in the Middle School Years: lEA's Third International Mathematics and Science Study (TIMSS). Chestnut Hill, 
MA: Boston College. 

7 Beaton, A.E., Mullis, I.V.S., Martin, M.O., Gonzalez, E.J., Kelly, D.L, and Smith, T.A. (1996). Mathematics 
Achievement in the Middle School Years: lEA's Third International Mathematics and Science Study (TIMSS). 
Chestnut Hill, MA: Boston College. 
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Table 1.3 



Gender Differences in Mathematics and Science Literacy Achievement for Students in Their 
Final Year of Secondary School* 



Country 


Males 


Females 


Difference 


TCI 


Gender Dil 


fferen< 


;e 




*’ • ■ Percent of 

A. Students 


Mean 

Achievement 


Percent of 
Students 


IIBBI 






Hungary 
2 Cyprus 
1 Lithuania 
f New Zealand 
Switzerland 


52 (2.5) 
45 (2.1) 
35 (3.0) 
49(1.7) 
56 (2.5) 


485 (4.5) 
456 (4.9) 
483 (6.7) 
540 (5.7) 
547(6.0) 


48 (2.5) 
55(2.1) 
65 (3.0) 
51 (1.7) 
44 (2.5) 


468 (4.5) 
439 (3.0) 
456 (7.4) 
511 (5.5) 
511 (7.5) 


17 (6.3) 
18(5.8) 

27 (10.0) 

28 (7.9) 
37 (9.6) 


65% 

48% 

43% 

70% 

82% 


1 




3 




Females 

Score 

Higher 


Males 

Score 

Higher 












2 Russian Federation 
Sweden 
Czech Republic 


38(1.0) 
49 (2.5) 
51 (5.1) 


499 (5.9) 
579 (5.8) 

500 (9.9) 


62 (1 .0) 
51 (2.5) 
49(5.1) 


462 (6.5) 
533 (3.6) 
452(13.8) 


37 (8.8) 
46 (6.8) 
48(17.0) 


48% 

71% 

78% 








S 


1 

■ 




| Countries Not Satisfying Guidelines for Sample Participation Rates (See Appendix B for Details) | 


Australia 
2 Austria 
Canada 
France 
Iceland 


42 (2.9) 
39(3.2) 
47(1.4) 

47 (3.1) 

48 (0.8) 


543(10.7) 
549 (7.8) 
544 (3.4) 
526 (5.9) 
565 (2.9) 


58 (2.9) 
61 (3.2) 
53(1.4) 
53 (3.1) 
52 (0.8) 


511 (9.3) 
502 (5.5) 
511 (3.4) 
487(4.8) 
522(1.9) 


32 (14.2) 
47 (9.6) 

33 (4.8) 
38 (7.6) 
43 (3.5) 


68% 

76% 

70% 

84% 

55% 










■ 




1 Italy 
Norway 
United States 


46 (3.3) 
51 (2.0) 
50(1.3) 


492 (6.9) 
564 (5.0) 
479 (4.2) 


54 (3.3) 

49 (2.0) 

50 (1 .3) 


461 (5.7) 
507 (4.5) 

462 (3.5) 


31 (8.9) 
57 (6.8) 
17(5.5) 


52% 

84% 

63% 


.... .. i 






jfj 


- 




| Countries With Unapproved Student Sampling (See Appendix B for Details ) | 


f Germany | 56 (5.2) 


512(8.2) | 44(5.2) | 


479(8.5) | 32(11.8) | 75% | 


1 IMN I 


— r 


| Countries With Unapproved Sampling Procedures and Low Participation Rates (See Appendix B for Details) \ 


Denmark 
2 Netherlands 
Slovenia 
South Africa 


45 (2.0) 
52 (2.3) 
51 (3.3) 
49 (1 .6) 


554 (4.5) 
584 (5.5) 
538(12.6) 
366(10.3) 


55 (2.0) 
48 (2.3) 
49(3.3) 
51 (1.6) 


507 (3.7) 
533 (5.9) 
492 (7.1) 
341 (11.8) 


47 (5.8) 
51 (8.0) 
46(14.4) 
25(15.7) 


58% 

78% 

88% 

49% 
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120 80 40 0 40 80 120 

International Averages 

Males Females Difference 

519 483 36 

(Averages of All Country Means) 

— SOURCE: IEA Third International Mathematics and Science Study (TIMSS), 1995-96. 



Gender difference statistically significant at .05 level. 
□ Gender difference not statistically significant. 
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* See Appendix A for characteristics of students tested. 

* Met guidelines for sample participation rates only after replacement schools were included (see Appendix B for details). 

1 National Desired Population does not cover all of International Desired Population (see Table B.4). 

2 National Defined Population covers less than 90 percent of National Desired Population (see Table B.4). 

( ) Standard errors appear in parentheses. Because results are rounded to the nearest whole number, some differences may appear inconsistent. 
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— Chapter 2 



Achievement in Mathematics Literacy and 
Science Literacy 



This chapter presents data summarizing achievement separately on the mathematics 
literacy scale and the science literacy scale. The mathematics literacy items address 
number sense, including fractions, percentages, and proportionality. Algebraic 
sense, measurement, and estimation are also covered, as are data representation 
and analysis. Several of the items emphasize reasoning and social utility. A 
general criterion in selecting the items was that they should involve the types of 
mathematics questions that could arise in real-life situations and that they be 
contextualized accordingly. Similarly, the science items selected for the literacy 
test were organized according to three areas of science - earth science, life science, 
and physical science - and included a reasoning and social utility component. 
Again, the emphasis was on trying to measure how well students can use their 
knowledge in addressing real-world problems having a science component. For 
both the mathematics literacy and science literacy items, students were permitted to 
use a calculator if they wished (see Chapter 4 for students’ reports on calculator use). 

Following the discussion in this chapter of average achievement in mathematics 
literacy and science literacy, Chapter 3 contains further information about the types 
of mathematics and science items, including seven example items for each area 
and the percentage of correct responses on those items for each TIMSS country. 

As we have seen in Chapter 1, there are differences in achievement among the 
participating countries on the TIMSS mathematics and science literacy test. Given 
that the test was designed to include mathematics and science items, it is interesting 
to examine whether the participating countries have particular strengths or weaknesses 
in their achievement in one or the other of the two areas. Thus, this chapter presents 
the results for the mathematics and science scales that formed the basis for the 
average composite results presented in Chapter 1. 





43 





CHAPTER 2 



o 

ERJC 


How Does Performance Compare Between the Mathematics and 
Science Areas? 

Table 2.1 presents the achievement results for the mathematics literacy scale. It 
shows the mean achievement for each country and the distribution of student 
performance within each country. Countries with a triangle pointing up performed 
above the international average of 500, those with a dot performed about the 
same as the international average, and those with triangles pointing down performed 
below the international average. The countries conforming to the TIMSS sampling 
guidelines and performing above the international average in mathematics literacy 
included Sweden, Switzerland, and New Zealand. Austria, Canada, France, Iceland, 
Norway, Denmark, and the Netherlands also achieved above the international average, 
although they encountered various difficulties in their sampling. The countries 
performing below the international average were Hungary, the Russian Federation, 
Lithuania, Cyprus, Italy, the United States, and South Africa. 

Figure 2.1 provides the information for comparing mean mathematics achievement 
between countries. This figure shows whether or not the differences in mean 
achievement between pairs of countries are statistically significant. The top-performing 
countries in mathematics literacy included the Netherlands, Sweden, Denmark, and 
Switzerland; both Sweden and Switzerland met the sampling guidelines. Iceland, 
Norway, France, Australia, New Zealand, Canada, Austria, and Slovenia all tended 
to perform similarly to Switzerland and to each other. However, of these countries, 
only New Zealand met the TIMSS sampling guidelines. 

Table 2.2 and Figure 2.2 show the corresponding results for the science literacy scale. 
Table 2.2 reveals that of the countries meeting the TIMSS sampling requirements, 
Sweden, New Zealand, and Switzerland performed above the international average 
(triangles pointing up). This parallels the findings in mathematics literacy. Other 
countries performing above the international average were Austria, Canada, Iceland, 
Norway, and the Netherlands. The countries performing below the international 
average in science literacy (triangle pointing down) included the Russian Federation, 
Hungary, Lithuania, Cyprus, Italy, the United States, and South Africa. 

The country comparison chart (Figure 2.2) shows that the countries with the highest mean 
achievement in science literacy were Sweden, the Netherlands, Iceland, and Norway, 
with only Sweden meeting the TIMSS sampling guidelines. Canada, New Zealand, 
and Australia performed similarly to Norway and to each other, with New Zealand 
meeting the sampling guidelines. Switzerland, which met the sampling guidelines, 
achieved at about the same level as Canada, New Zealand, and Australia, as did 
Austria and Slovenia. 

Table 2.3 compares performance in mathematics and science literacy. It presents mean 
literacy achievement separately for mathematics and science, as well as the difference 
between the means. The last column shows the amount of the difference, whether 
its direction favors mathematics or science, and whether it is statistically significant 
(darkened bar). Regardless of direction, the differences between mathematics and 
science literacy were small or negligible in nearly half of the countries. However, 
Lithuania, Hungary, Switzerland, France, and Denmark performed significantly higher 
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in mathematics literacy than in science literacy. In contrast, Sweden, the Russian 
Federation, the Czech Republic, Canada, Iceland, Norway, and the United States 
had significantly higher achievement in science literacy than in mathematics literacy. 

Table 2.4 shows the differences in mathematics literacy performance by gender, 
and Table 2.5 presents the corresponding gender differences for science literacy. 
The results differ somewhat from the patterns noted in TIMSS at the eighth grade, 
where gender differences favoring males were found in both mathematics and science 
but the differences were more pervasive in science. For students in their final year 
of school, the gender differences favoring males are significant in mathematics as 
well as science in most countries. In mathematics literacy, most of the countries 
showed gender differences favoring males, although these were not statistically 
significant in Hungary, the United States, and South Africa. All countries except 
South Africa showed statistically significant gender differences in science literacy 
favoring males. Thus, it appears that as students leave school the achievement 
differences favoring males are found nearly equally in mathematics and science literacy. 
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Table 2.1 



Distributions of Achievement in Mathematics Literacy for Students in Their 
Final Year of Secondary School* 



Country 


Mean 


TCI 


Sweden 


▲ 552(4.3) 


71% 


Switzerland 


▲ 540(5.8) 


82% 


f New Zealand 


▲ 522(4.5) 


70% 


Hungary 


▼ 483(3.2) 


65% 


2 Russian Federation 


▼ 471 (6.2) 


48% 


1 Lithuania 


▼ 469(6.1) 


43% 


Czech Republic 


• 466(12.3) 


78% 


2 Cyprus 


▼ 446(2.5) 


48% 



Mathematics Literacy Achievement Scale Score 



C I, 
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Countries Not Satisfying Guidelines for Sample Participation Rates (See Appendix B for Details) 



Australia 


• 522(9.3) 


68% 


2 Austria 


▲ 518(5.3) 


76% 


Canada 


▲ 519(2.8) 


70% 


France 


▲ 523(5.1) 


84% 


Iceland 


▲ 534(2.0) 


55% 


1 Italy 


▼ 476(5.5) 


52% 


Norway 


▲ 528(4.1) 


84% 


United States 


t 461 (3.2) 


63% 




Countries With Unapproved Student Sampling (See Appendix B for Details) 



* Germany 



• 495(5.9) 



75% 



Countries With Unapproved Sampling Procedures and Low Participation Rates (See Appendix B for Details) 



Denmark 


a 547(3.3) 


58% 


2 Netherlands 


a 560(4.7) 


78% 


Slovenia 


• 512(8.3) 


88% 


South Africa 


▼ 356(8.3) 


49% 
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International Average = 500 
(Average of All Country Means) 



Mean and Confidence Interval (±2SE) 



▲ = Country mean significantly higher than international mean 
▼ = Country mean significantly lower than international mean 

• = No statistically significant difference between country mean and international mean 



* See Appendix A for characteristics of students sampled. 

1 Met guidelines for sample participation rates only after replacement schools were included (see Appendix B for details). 

1 National Desired Population does not cover all of International Desired Population (see Table B.4). 

2 National Defined Population covers less than 90 percent of National Desired Population (see Table B.4). 

( ) Standard errors appear in parentheses. Because results are rounded to the nearest whole number, some totals may appear inconsistent. 
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Multiple Comparisons of Mathematics Literacy Achievement for Students in 
Their Final Year of Secondary School* 

Instructions: Read across the row for a country to compare performance with the countries listed in the heading of the chart. 
The symbols indicate whether the mean achievement of the country in the row is significantly lower than that of the comparison 
country, significantly higher than that of the comparison country, or if there is no statistically significant difference between the 
two countries.* 
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Countries are ordered by mean achievement across the heading and down the rows. 



H Mean achievement 

significantly higher than 
comparison country 




No statistically significant 
difference from comparison 
country 




Mean achievement 
significantly lower than 
comparison country 



* See Appendix A for characteristics of the students sampled. 

* Statistically significant at .05 level, adjusted for multiple comparisons. 

Countries shown in italics did not satisfy one or more guidelines for sample participation rates or student sampling (see Figure B.4). 



o 

ERIC 



63 



47 



SOURCE: IEA Third International Mathematics and Science Study (TIMSS), 1995-96. 
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Table 2.2 



Distributions of Achievement in Science Literacy for Students in Their 
Final Year of Secondary School* 



Science Literacy Achievement Scale Score 



Country 


Mean 


TCI 


Sweden 


a 559(4.4) 


71% 


New Zealand 


a 529(5.2) 


70% 


Switzerland 


a 523(5.3) 


82% 


Czech Republic 


• 487(8.8) 


78% 


Russian Federation 


▼ 481 (5.7) 


48% 


Hungary 


▼ 471 (3.0) 


65% 


Lithuania 


▼ 461 (5.7) 


43% 


Cyprus 


▼ 448(3.0) 


48% 



i I 



1 , T 



i — r 



L 1 



i t ; 



Countries Not Satisfying Guidelines for Sample Participation Rates (See Appendix B for Details) 



Australia 
! Austria 
Canada 
France 
Iceland 



527(9.8) 
520 (5.6) 
532 (2.6) 
487 (5.1) 
549(1.5) 



1 Italy 
Norway 
United States 



▼ 475(5.3) 
▲ 544(4.1) 

▼ 480(3.3) 



68% 

76% 

70% 

84% 

55% 



52% 

84% 

63% 



m 



T 



3 



Countries With Unapproved Student Sampling (See Appendix B for Details) 



1 Germany 



• 497(5.1) | 75% 



Countries With Unapproved Sampling Procedures and Low Participation Rates (See Appendix B for Details) 



Denmark 


• 509(3.6) 


58% 


2 Netherlands 


a 558(5.3) 


78% 


Slovenia 


• 517(8.2) 


88% 


South Africa 


t 349(10.5) 


49% 



r 



X 



i — r 



200 250 300 350 400 450 500 550 600 650 700 750 800 



| Percentiles of Performance — j 

5th 25th 75th 95th 

t = l ■ i — i 



International Average = 500 
(Average of All Country Means) 



Mean and Confidence Interval (±2SE) 



▲ = Country mean significantly higher than international mean 
▼ = Country mean significantly lower than international mean 

• = No statistically significant difference between country mean and international mean 



* See Appendix A for characteristics of students sampled. 

f Met guidelines for sample participation rates only after replacement schools were included (see Appendix B for details). 

1 National Desired Population does not cover all of International Desired Population (see Table B.4). 

2 National Defined Population covers less than 90 percent of National Desired Population (see Table B.4). 

( ) Standard errors appear in parentheses. Because results are rounded to the nearest whole number, some totals may appear inconsistent. 
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SOURCE: IEA Third International Mathematics and Science Study (TIMSS), 1995-96. 
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Multiple Comparisons of Science Literacy Achievement 
for Students in Their Final Year of Secondary School* 

Instructions: Read across the row for a country to compare performance with the countries listed in the heading of the chart. 
The symbols indicate whether the mean achievement of the country in the row is significantly lower than that of the comparison 
country, significantly higher than that of the comparison country, or if there is no statistically significant difference between the 
two countries. 1 " 
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Countries are ordered by mean achievement across the heading and down the rows. 




Mean achievement 
significantly higher than 
comparison country 




No statistically significant 
difference from comparison 
country 




Mean achievement 
significantly lower than 
comparison country 



* See Appendix A for characteristics of the students sampled. 

1 Statistically significant at .05 level, adjusted for multiple comparisons. 

Countries shown in italics did not satisfy one or more guidelines for sample participation rates or student sampling (see Figure B.4). 



o 

ERIC 



65 



49 



SOURCE: IEA Third international Mathematics and Science Study (TIMSS), 1995-96. 
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Table 2.3 



Differences in Performance Between Mathematics Literacy and Science Literacy for 
Students in Their Final Year of Secondary School* 



Country 


Mathematics 


Science 




Literacy Mean 


Literacy Mean Difference 


Subject Difference 




Score 


Score 



2 Cyprus 
f New Zealand 
Sweden 

1 Lithuania 

2 Russian Federation 


446 (2.5) 
522 (4.5) 
552 (4.3) 
469 (6.1) 
471 (6.2) 


448 (3.0) 
529 (5.2) 
559 (4.4) 
461 (5.7) 
481 (5.7) 


2 (2.4) 
7 (2.8) 
7(1.3) 
9 (2.2) 
10(2.5) 


■ 


Science 

Higher 


t 

nz 




■ 


Mathematics 

Higher 










Hungary 


483(3.2) 


471 (3.0) 


13(1.3) 








mm 






Switzerland 


540 (5.8) 


523 (5.3) 


18(2.3) 








mmm 






Czech Republic 


466(12.3) 


487 (8.8) 


20 (4.1) 




1 


mmam 








| Countries Not Satisfying Guidelines for Sample Participation Rates (See Appendix B for Details) 


Australia 


522 (9.3) 


527 (9.8) 


5 (2.4) 






■c 








2 Austria 


518(5.3) 


520 (5.6) 


2(2.1) 






c 








Canada 

Franrp 


519(2.8) 
APT fA “h 


532 (2.6) 

4ft7 ( A 1 'l 


13(1.7) 

Ofi /O Q\ 














1 1 CM IV/C 

Iceland 


534 (2.0) 


HOf yu. 1 f 

549(1.5) 


15(1.7) 














1 Italy 


476 (5.5) 


475 (5.3) 


1 (2.2) 














Norway 


528 (4.1) 


544 (4.1) 


16(1.8) 














United States 


461 (3.2) 


480 (3.3) 


19 (1.5) 






■m 









| Countries With Unapproved^ Student Sampling (See "Appendix B for Details) 



[ Germany 495(5.9) | 497(5.1) 2(2.4) f j | |j 

| Countries With Unapproved^ Sampling Procedures and Low Participation Rates (See Appendix B for Details) ' 



Denmark 


547 (3.3) 


509 (3.6) 


38 (2.8) 


2 Netherlands 


560 (4.7) 


558 (5.3) 


2 (2.3) 


Slovenia 


512(8.3) 


517(8.2) 


6 (2.3) 


South Africa 


356 (8.3) 


349(10.5) 


LM 



SOURCE: IEA Third International Mathematics and Science Study (TIMSS), 1995-9 



60 



40 



20 



20 



40 



60 



H Difference statistically significant at .05 level. 
:□ Difference not statistically significant. 



* See Appendix A for characteristics of students sampled. 

* Met guidelines for sample participation rates only after replacement schools were included (see Appendix B for details). 

1 National Desired Population does not cover all of International Desired Population (see Table B.4). 

2 National Defined Population covers less than 90 percent of National Desired Population (see Table B.4). 

( ) Standard errors appear in parentheses. Because results are rounded to the nearest whole number, some differences may appear inconsistent. 
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Achievement in Mathematics Literacy by Gender for Students in Their Final Year of 
Secondary School* 



Males 


Females 


Difference 


TCI 


Gender Dif 


f0fenc 


e 




uuui ili y 


Percent of 
Students 


Wean 

Achievement 


Percent of 1 Mean **$3$ 

Students Achievement |V 7 ^ T . 7E* c /•/ 




•i. ■ ^ ; , i ' ■ s 


Hungary 
2 Cyprus 

1 Lithuania 

f New Zealand 

2 Russian Federation 


52 (2.5) 
45 (2.1) 
35 (3.0) 
49(1.7) 
38(1.0) 


485 (4.9) 
454 (4.9) 
485 (7.3) 
536 (4.9) 
488 (6.5) 


48 (2.5) 
55(2.1) 
65 (3.0) 
51 (1.7) 
62 (1 .0) 


481 (4.8) 
439 (3.7) 
461 (7.7) 
507 (6.2) 
460 (6.6) 


5 (6.9) 
15(6.1) 
23(10.6) 
29 (7.9) 
27 (9.2) 


65% 

48% 

43% 

70% 

48% 


Females 

Score 

Higher 






1 

Males 

Score 

Higher 












Switzerland 
Sweden 
Czech Republic 


56 (2.5) 
49 (2.5) 
51 (5.1) 


555 (6.4) 
573 (5.9) 
488(11.3) 


44 (2.5) 
51 (2.5) 
49 (5.1) 


522 (7.4) 
531 (3.9) 
443(16.8) 


33 (9.8) 
42 (7.0) 
45 (20.2) 


82% 

71% 

78% 










1 

1 




I Countries Not Satisfying Guidelines for Sample Participation Rates (See Appendix B for Details 





Australia 
2 Austria 
Canada 
France 
Iceland 


42 (2.9) 

39 (3.2) 
47(1.4) 
47(3.1) i 
48(0.8) ' 


540(10.3) 
545 (7.2) 
537 (3.8) 
544 (5.6) 
558 (3.4) 


58 (2.9) 
61 (3.2) 
53 (1 .4) 
53(3.1) 
52 (0.8) 


510(9.3) 

503 (5.5) 

504 (3.5) 
506 (5.3) 
514(2.2) 


30(13.9) 
41 (9.0) 
34 (5.2) 
38 (7.7) 
44 (4.1) 


68% 

76% 

70% 

84% 

55% 










1 




1 Italy 
Norway 
United States 


46(3.3) 
51 (2.0) 
50(1.3) 


490(7.4) 
555 (5.3) 
466 (4.1) 


54 (3.3) 
49 (2.0) 
50(1.3) 


464 (6.0) 
501 (4.8) 
456(3.6) 


26 (9.5) 
54 (7.1) 
11 (5.5) 


52% 

84% 

63% 










■ 




I Countries With Unapproved Student Sampling (See Appendix B for Details) I 


f Germany 1 56 (5.2) 


| 509 (8.8) | 44(5.2) 


| 480 (8.8) | 29(12.4)| 75% | 






■ 




t 


1 Countries With Unapproved Sampling Procedures and Low Participation Rates (See Appendix B for Details) \ 


Denmark 
2 Netherlands 
Slovenia 
South Africa 


45 (2.0) 
52 (2.3) 
51 (3.3) 
49(1.6) 


575 (4.0) 
585 (5.6) 
535(12.7) 
365 (9.3) 


55 (2.0) 

48 (2.3) 

49 (3.3) 
51 (1.6) 


523 (4.0) 
533 (5.9) 
490 (8.0) 
348(10.8) 


52 (5.7) 

53 (8.2) 
46 (15.0) 
17 (14.3) 


58% 

78% 

88% 

49% 








3 


L 





SOURCE: IEA Third International Mathematics and Science Study (TIMSS), 1995-96. 120 80 40 0 40 80 120 



International Averages 
Males Females Difference 

518 485 33 

(Averages of All Country Means) 




Gender difference statistically significant at .05 level. 
Gender difference not statistically significant 



* See Appendix A for characteristics of students sampled. 

t Met guidelines for sample participation rates only after replacement schools were included (see Appendix B for details). 

1 National Desired Population does not cover all of International Desired Population (see Table B.4). 

2 National Defined Population covers less than 90 percent of National Desired Population (see Table B.4). 

( ) Standard errors appear in parentheses. Because results are rounded to the nearest whole number, some differences may appear inconsistent. 
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BEST COPY AVAILABLE 



67 



51 
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Table 2.5 



Achievement in Science Literacy by Gender for Students in Their Final Year of 
Secondary School* 



Males 



Females 



Difference TCI 



Gender Difference 



IC ?! Percent of 

1 3* ! Students 


Mean 

Achievement 


Percent of 
Students 


Mean 

Achievement 




2 Cyprus 


45 (2.1) 


459 (5.8) 


55 (2.1) 


439 (3.0) 


20 (6.5) 


48% 


f New Zealand 


49(1.7) 


543(7.1) 


51 (1.7) 


515(5.2) 


28 (8.8) 


70% 


Hungary 


52 (2.5) 


484 (4.2) 


48 (2.5) 


455 (4.3) 


29 (6.0) 


65% 


1 Lithuania 


35 (3.0) 


481 (6.4) 


65 (3.0) 


450 (7.3) 


31 (9.7) 


43% 


Switzerland 


56 (2.5) 


540 (6.1) 


44 (2.5) 


500 (7.8) 


40 (9.9) 


82% 


2 Russian Federation 


38(1.0) 


510(5.7) 


62(1.0) 


1 463 (6.7) 


47 (8.8) 


48% 


Sweden 


49 (2.5) 


585 (5.9) 


51 (2.5) 


534 (3.5) 


50 (6.8) 


71% 


Czech Republic 


51 (5.1) 


512(8.8) 


49 (5.1) 


460(11.0) 


51 (14.0) 


78% 



Females 

Score 

Higher 




Males 

Score 

Higher 




1 Countries Not Satisfyin, 


g Guidelines for Sample Participation Rates (See Appendix B for Details) | 


Australia 
2 Austria 
Canada 
France 
Iceland 


42 (2.9) 
39 (3.2) 
47(1.4) 

47 (3.1) 

48 (0.8) 


547(11.5) 
554 (8.7) 
550 (3.6) 
508 (6.7) 
572 (2.7) 


58 (2.9) 
61 (3.2) 
53(1.4) 
53 (3.1) 
52 (0.8) 


513(9.4) 
501 (5.8) 
518(3.8) 
468 (4.8) 
530 (2.1) 


34(14.8) 
53(10.4) 
32 (5.2) 
39 (8.3) 
41 (3.4) 


68% 

76% 

70% 

84% 

55% 


* * 




- 


| 


■ 




1 Italy 
Norway 
United States 


46 (3.3) 
51 (2.0) 
50(1.3) ! 


495 (6.7) 
574 (5.1) 
492 (4.5) 


54 (3.3) 
49(2.0) 
50(1.3) l 


458 (5.6) 
513(4.5) 
469 (3.9) 


37 (8.8) 
61 (6.8) 
23 (5.9) 


52% 

84% 

63% 


, * * 1 






~ 


- 




| Countries With Unapproved Student Sampling (See Appendix B for Details) 1 


f Germany | 56 (5.2) | 


514(7.9) | 44(5.2) | 


| 478(8.5) | 35(11.6) | 75% | * •* | 






J 




| Countries With Unapproved Sampling Procedures and Low Participation Rates (See A 


ppendix B for Details) | 


Denmark 
2 Netherlands 
Slovenia 
South Africa 


45 (2.0) 
52 (2.3) 
51 (3.3) 
49(1.6) 


532 (5.4) 
582 (5.7) 
541 (12.7) 
367(11.5) 


55 (2.0) 

48 (2.3) 

49 (3.3) 
51 (1.6) 


490 (4.1) 
532 (6.2) 
494 (6.4) 
333(13.0) 


41 (6.8) 
49 (8.4) 
47(14.3) 
34 (17.4) 


58% 

78% 

88% 

49% 


? * 


» H » • 






■ 

■ ' * ; 


j * * 



SOURCE: IEA Third International Mathematics and Science Study (TIMSS), 1995-9 
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80 



40 



40 



80 



120 



International Averages 
Males Females Difference 

521 482 39 

(Averages of All Country Means) 



H Gender difference statistically significant at .05 level. 
□ Gender difference not statistically significant. * . 



* See Appendix A for characteristics of students sampled. 

t Met guidelines for sample participation rates only after replacement schools were included (see Appendix B for details). 

1 National Desired Population does not cover all of International Desired Population (see Table B.4). 

2 National Defined Population covers less than 90 percent of National Desired Population (see Table B.4). 

( ) Standard errors appear in parentheses. Because results are rounded to the nearest whole number, some differences may appear inconsistent. 
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CHAPTER 2 



How Does Final-Year Performance in Secondary School Compare 
with Eighth-Grade Performance? 

Achievement for students in the final year of secondary school was estimated separately 
from achievement at the middle school grades. That is, different tests were used and 
different content areas emphasized. Therefore, the scale scores are not comparable, 
and no direct comparison can be made between the performance of the upper secondary 
school students and that of the eighth-grade students. One way to gauge relative 
performance at the two levels, however, is to compare a country’s performance with 
the international mean at each of the two points in school. For example, for the countries 
participating in both the middle school and upper secondary school testing, mean 
mathematics achievement in comparison with the international average is portrayed 
in Figure 2.3, with the eighth-grade results for each country derived from 
Mathematics in the Middle School Years: lEA’s Third International Mathematics 
and Science Study' and the results for the final year of secondary school taken from 
Table 2. 1 of the present report. 

As shown in Figure 2.3, Switzerland, the Netherlands, Austria, France, and Canada 
were above the international average both at the eighth grade and for their upper 
secondary school students. However, the countries ranking high in mathematics 
achievement at the eighth grade did not always rank high in mathematics literacy at 
the upper secondary level. The Czech Republic, Slovenia, and Australia were above 
the international average at the eighth grade, but at about the international average 
for upper secondary school students. Hungary and the Russian Federation performed 
above the international average at the eighth grade but below it for the final year of 
secondary school. The United States performed about at the international average at 
the eighth grade, but below it for upper secondary school students. Conversely, 
Sweden, New Zealand, and Denmark performed similarly to the international average 
at the eighth grade, but above it at the upper secondary level, while Norway and 
Iceland moved from below the international average at the eighth grade to above it 
for upper secondary school students. 

Figure 2.4 shows the results for science achievement relative to the international 
average at the eighth grade and for science literacy at the upper secondary school level. 
The eighth-grade results for countries also participating in the science testing of 
students in the eighth grade were derived from Science Achievement in the Middle 
School Years: lEA’s Third International Mathematics and Science Study. 2 



1 Beoton, A.E., Mullis, I.V.S., Morlin, M.O., Gonzolez, E.J., Kelly, D.L., ond Smith, T.A. (1996). Mathematics 
Achievement in the Middle School Years: lEA's Third International Mathematics and Science Study ITIMSS). 
Chestnut Hill, MA: Boston College. 

2 Beoton, A.E., Mortin, M.O., Mullis, I.V.S., Gonzolez, E.J., Smith, T.A., ond Kelly, D.L. (1996). Science 
Achievement in the Middle School Years: lEA's Third International Mathematics and Science Study ( TIMSSj . 

© Chestnut Hill, MA: Boston College. 
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Just as with the mathematics results, the high-ranking countries were not the same 
for the eighth grade and the final year of secondary school. Although the Netherlands, 
Austria, Sweden, Canada, and Norway were above the international average at both 
levels, the Czech Republic, Slovenia, Australia, and Germany moved toward the 
international average at the upper secondary level and the Russian Federation and 
the United States moved below it. In contrast, New Zealand and Switzerland performed 
at about the international average at the eighth grade, but above it at the upper 
secondary level. Iceland moved from below the international average at the eighth 
grade to above it at the upper secondary level, while France and Denmark moved 
from below the international average to about the international average. 

In reading Figures 2.3 and 2.4, however, it is important to remember that the scales 
for the eighth grade and the upper secondary level are not directly comparable. For 
example, looking at the international averages, it cannot be said that the eighth-grade 
students as a whole outperformed the students in their final year of secondary school. 
Since seventh and eighth graders were given the same mathematics and science tests, 
the international average of the TIMSS scales for the two grades combined was set 
at 500. As would be expected, the eighth graders outperformed the seventh graders, 
resulting in a mean somewhat higher than 500 (i.e., 5 1 1 in mathematics and 5 15 in 
science, as shown in Figures 2.3 and 2.4, respectively). Using the same approach, 
the international average for the secondary school students also was arbitrarily set at 
500. Therefore, the differences in the international means between the eighth grade 
and the final year of secondary school are simply an artifact of the scaling procedures 
used. Also, note that the international averages shown for the eighth grade in 
Figures 2.3 and 2.4 will not match those reported previously for all 41 countries 
participating at the eighth grade, because they are based only on the 20 countries 
that also participated in the testing of students in their final year of secondary school. 
(Even though Italy’s results are contained in the present report, its eighth-grade 
results were not available.) 
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Figure 2.3 



Mathematics Performance at Eighth Grade + and Final Year of Secondary School* 
Compared with the International Averages 



Eighth Grade 
Mathematics Scale 




Final Year of Secondary School 
Mathematics Literacy Scale 


Country 


Difference 

from 

International 

Average 




Country 


TCI 


Difference 

from 

International 

Average 


Czech Republic 


53 (4.9) 




Netherlands 


78% 


60 (4.7) ; 


Switzerland 


35 (2.8) 




Sweden 


71% 


52 (4.3) ! 


Netherlands 


30 (6.7) 




Denmark 


58% 


47 (3.3) | 


Slovenia 


30 (3.1) 




Switzerland 


82% 


40 (5.8) 


Austria 


29 (3.0) 




Iceland 


55% 


34(2.0) 


France 


27 (2.9) 




Norway 


84% 


28 (4.1) 


Hungary 


27 (3.2) 




France 


84% 


23 (5.1) 


Russian Federation 


25 (5.3) 




New Zealand 


70% 


22 (4.5) 


Australia 


19(4.0) 




Canada 


70% 


19(2.8) 


Canada 


17(2.4) 




Austria 


76% 


18(5.3) 


Sweden 


8 (3.0) 




Australia 


68% 


22 (9.3) 


Germany 


-1 (4.5) 




Slovenia 


88% 


12(8.3) 


New Zealand 


-3 (4.5) 




Germany 


75% 


-5 (5.9) 


Denmark 


-8 (2.8) 




Czech Republic 


78% 


-34 (12.3) 


United States 


•11 (4.6) 




Hungary 


65% 


-17(3.2) 


Norway 


-7 (2.2) 




Russian Federation 


48% 


-29 (6.2) 


Iceland 


-24 (4.5) 




Lithuania 


43% 


-31 (6.1) 


Lithuania 


-33 (3.5) 




United States 


63% 


-39 (3.2) 


Cyprus 


-37 (1.9) 




Cyprus 


48% 


-54 (2.5) 


South Africa 


-157 (4.4) 




South Africa 


49% 


-144 (8.3) 


International Average 

(Average of All Country Means) 


511 (0.8) 




International Average 

(Average of All Country Means) 




500(1.3) 



SOURCE: IEA Third International Mathematics and Science Study (TIMSS), 1995-96. 



Q Significantly Higher than International Average 
Q Not Significantly Different from International Average 
Q Significantly Lower than International Average 



* Eighth grade in most countries. 

* See Appendix A for characteristics of the students sampled. 

( ) Standard errors appear in parentheses. Because results are rounded to the nearest whole number, some totals may appear inconsistent. 
Countries shown in italics did not satisfy one or more guidelines for sample participation rates or student sampling procedures (see Figure B.4). 

Includes countries that participated in TIMSS testing at both eighth grade and final year of secondary school. The eighth-grade results are derived 
from those reported in Mathematics Achievement in the Middle School Years: lEA's Third International Mathematics and Science Study. 
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Figure 2.4 



Science Performance at Eighth Grade + and Final Year of Secondary School* Compared 
with the International Averages 



Eighth Grade 


Science Scale 


Country 


Difference 

from 

International 

Average 


Czech Republic 


59 (4.3) 


Netherlands 


45 (5.0) 


Slovenia 


45 (2.5) 


1 Austria 


43 (3.7) 


| Hungary 


39 (2.8) 


Australia 


30 (3.9) 


Russian Federation 


23 (4.0) 


j Sweden 


20 (3.0) 


United States 


20 (4.7) 


Germany 


16(4.8) 


Canada 


16(2.6) 


j Norway 


12(1.9) 


New Zealand 


11 (4.4) 


Switzerland 


7 (2.5) 


France 


-17(2.5) 


Iceland 


-21 (4.0) 


Denmark 


-37 (3.1) 


Lithuania 


-38 (3.4) 


Cyprus 


-52 (1.9) 


South Africa 


-189 (6.6) 


International Average 

(Average of All Country Means) 


515(0.8) 



Final Year of Secondary School 


Science Literacy Scale 


Country 


TCI 


Difference 

from 

International 

Average 


Sweden 


71% 


59 (4.4) ' 


Netherlands 


78% 


58(5.3) 


Iceland 


55% 


49(1.5) , 


Norway 


84% 


44(4.1) , 


Canada 


70% 


32 (2.6) 


New Zealand 


70% 


29(5.2) ; 


Switzerland 


82% 


23(5.3) i 


Austha 


76% 


20(5.6) ; 


Australia 


68% 


27 (9:8) 


j Slovenia 


88% 


17(8.2) 


\ Denmark 


58% 


9 (3.6) 


j Germany 


75% 


-3 (5.1) 


France 


84% 


-13 (5.1) 


Czech Republic 


78% 


-13 (8.8) 


Russian Federation 


48% 


-19 (5.7) 


United States 


63% 


-20 (3.3) 


Hungary 


65% 


-29 (3.0) 


Lithuania 


43% 


-39 (5.7) 


Cyprus 


48% 


-52 (3.0) 


South Africa 


49% 


-151 (10.5) 


International Average 

(Average of All Country Means) 




500(1.3) 



SOURCE: IEA Third International Mathematics and Science Study (TIMSS), 1995-96. 



Q Significantly Higher than International Average 
Q Not Significantly Different from International Average 
Q Significantly Lower than International Average 



1 Eighth grade in most countries. 

* See Appendix A for characteristics of the students sampled. 

( ) Standard errors appear in parentheses. Because results are rounded to the nearest whole number, some totals may appear inconsistent. 
Countries shown in italics did not satisfy one or more guidelines for sample participation rates or student sampling procedures (see Figure B.4). 

Includes countries that participated in TIMSS testing at both eighth grade and final year of secondary school. The eighth-grade results are derived 
from those reported in Science Achievement in the Middle School Years: lEA's Third International Mathematics and Science Study. 
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— Chapter 3 



Performance on Mathematics and Science 
Literacy Example Items 



This chapter presents seven example test questions in the mathematics literacy area 
and seven in the science literacy area, and performance on each of the 14 items for 
each of the TIMSS countries. The example items in this chapter were chosen to 
illustrate the different topics covered in each area, the different performance 
expectations, and the range of item formats used. To provide some sense of what 
types of items were answered correctly by higher-performing students as compared 
with lower-performing students, the items in each area span a range of difficulty. 
Finally, it should be noted that all these items and others have been released for 
use by the public. 1 

The presentation for each of the two subject areas begins with a brief description 
of the major topics included in that area, followed by seven tables showing achieve- 
ment on the example items. Each table presents the example item in its entirety 
and shows the percentages of correct responses for each of the TIMSS countries. 
The correct answer is circled for multiple-choice items and shown in the answer 
space for short-answer items. For extended-response questions, the answer shown 
exemplifies the types of student responses that were given full credit. All of the 
responses shown have been reproduced from students’ actual test booklets. The 
extended-response questions were scored using a method that provided partial credit 
for responses indicating some conceptual understanding by students, despite a 
lack of completeness. For these questions, the tables show the percentages of students 
receiving partial credit in each country as well as the percentages of those receiving 
full credit. 

The seven tables showing the country-by-country results on each item within the 
subject area are followed by a “difficulty map” relating achievement on each of 
the example items to performance on the TIMSS international mathematics literacy 
or science literacy scale. 




’ The IEA retained about 60% of the TIMSS items as secure for possible future use in measuring 
international trends in mathematics and science literacy achievement. All remaining items are available for 
general use. 
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What Are Some Examples of Performance in Mathematics 
Literacy? 



The items selected for mathematics literacy were designed to define the content area 
adequately, while restricting the test items to the few content areas most closely 
related to the notion of mathematical literacy. 2 The items represent the domains of 
number sense (including fractions and percentages as well as proportionality); 
algebraic sense; data representation; and measurement and estimation. Several items 
were designed to measure the component of reasoning and social utility in math- 
ematics. These items emphasize the types of understanding students will need for 
full participation in today’s technology-dependent, information-rich society. 

As shown in Table 3.1, final-year students in most countries selected the correct 
answer to the proportionality problem requiring calculating the number of calories 
in a portion of food (Example Item 1). The international average percentage of correct 
responses across the participating countries was 71%, with 80% or more of the 
students in the Netherlands and France answering correctly. 

Table 3.2 presents Example Item 2, asking students to determine the number of 
defective light bulbs in a batch on the basis of testing a sample. This proportionality 
task is set in the context of sampling, which students might encounter in quality- 
control procedures in the workplace, in opinion polling, or in market research. As 
with Example Item 1, final-year students in many countries did relatively well on 
this item (international average 66%). More than three-fourths of the students in 
New Zealand, Sweden, the Netherlands, and Slovenia selected the correct response. 

Example Item 3 was a two-part item, requiring students to interpret the information 
in a travel graph and respond in an open-ended format. The results are shown in 
Table 3.3. In part A of the item, which was relatively straightforward, students had 
to be able to read the line graph and use the labeled information on the vertical axis 
to provide the answer of 60 km per hour as the car’s maximum speed. Students were 
somewhat less successful with part B, which required interpretation of the information 
in the graph based on events and the ability to read a marked but unlabeled point on 
the horizontal axis. Whereas the international average was 74% correct responses on 
part A, only 59% of the final-year students, on average, provided the correct answer of 
9:07 for the time that Kelly slammed on her brakes (part B). About 7% of the 
students, on average, across countries responded that Kelly slammed on her brakes at 
9:06, the closest labeled point on the horizontal axis. 

Example Item 4 also asked final-year students to interpret the information in graphs. 
Students were given a bar graph presenting information about the yearly value of 
sales in Zedland of music cassettes, records, and CDs, and a pie graph showing the 
percentage of CD sales by purchasers’ age in 1992 (see Table 3.4). Students were 
asked to use the information in the two graphs to determine the amount of money 
spent by 12- to 19-year-olds in 1992, and to show their calculations. On average, 
44% of the students gave a fully correct response. A number of students responded 



2 For a full discussion of the mathematics literacy items, see Orpwood, G. and Garden, R.A. j 1998). Assessing 
Mathematics and Science Literacy ; TIMSS Monograph No. 4. Vancouver, B.C.: Pacific Educational Press. 
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correctly with 86.4 million zeds (720 million zeds x .12) supported by an explanation 
of how the answer was obtained. However, students did not need to read the bar 
concerned as representing exactly 720 million zeds; any number in the range of 700 
million to 730 million zeds was acceptable. For example, the answer shown for 
Example Item 4 used 715 million zeds. Thus any answer in the range of 84 million 
to 87.6 million zeds was given full credit as long as the method of obtaining it was 
appropriate. Sixty percent or more of the students provided fully correct responses 
to this question in Sweden, Switzerland, Denmark, and the Netherlands. Another 
one-fifth to two-fifths of the students in many countries received partial credit for 
their responses. The latter omitted the factor of 1 million, made a decimal or other 
computation error, or provided other calculations that indicated understanding of the 
problem even though the final answer was missing or incorrect. 

Example Item 5 is a multi-step measurement item involving volume and percentages. 
Students were asked about the increase in volume of a cube-shaped carton if each 
of its sides was increased by 10%. As revealed by the results in Table 3.5, this 
multiple-choice item was quite difficult for students in most countries. Except in the 
Netherlands, fewer than half the final-year students selected the correct answer in 
each of the participating TIMSS countries. Fewer than one-fifth answered correctly 
in the United States and South Africa. 

Example Item 6 is an open-ended question, asking students to explain whether a 
reporter’s statement about a “huge increase” was a reasonable interpretation of a graph 
showing the number of robberies per year. As shown in Table 3.6, on average 
approximately one-fifth of the students across countries received full credit for their 
responses. They did so by noting that only a small part of the graph is shown, that 
10 (2%) is not a very large increase over the whole, or that the graph is misleading for 
some other reason. Another one- fourth of the final-year students, on average, received 
partial credit for this problem. They disagreed with the reporter, or said that 10 was 
not a large increase but did not say why, or rejected the interpretation for other 
reasons not relevant to the task. More than 60% of the students in New Zealand, 
Sweden, Australia, and Iceland provided either fully or partially correct responses 
to this question. 

In Example Item 7, students were asked to sketch their own line graph. They were 
presented with a grid and asked to show the relationship between a person’s height 
and age from birth to 30 years. Students were specifically asked to label their graphs 
and to use a realistic height scale along that axis. To receive full credit, students 
needed to think out how such a graph might look, and then produce a reasonable sketch. 
Fully correct responses had correct scales and labels on both axes - Age 0 to 30 years 
and Height 0 to 200 cm (or 0 to 80 inches, or to 7 feet). Also, the line relating height 
to age needed to start at approximately 50 cm (20 inches) and reach a reasonable 
maximum at a realistic age (14 to 20 years), after which it would remain essentially 
flat. The results are presented in Table 3.7 for students providing fully correct and 
partially correct responses. Across the participating countries, approximately one-fifth 
of the final-year students drew fully correct graphs. On average, another 28% drew 
partially correct graphs. In graphs receiving partial credit, all except one of the features 
were correct. For example, partially correct graphs may have started the height 
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line incorrectly (e.g., with a height of zero), had an unrealistic age for maximum 
height, had the line decrease after its peak, or included incorrect scales or labels. 
More than 60% of the students in Canada and Norway received either full or partial 
credit for their graphs. 

Figure 3. 1 shows the relationship between performance on the TIMSS international 
mathematics literacy scale and achievement on the seven example items in the 
mathematics literacy area. 3 The international achievement on each example item 
is indicated by both the average percentage correct across all countries and the 
international mathematics literacy scale value, or item difficulty level, for each item. 

For the figure, the item results have been placed on the scale at the point where 
students at that level were more likely than not (65% probability) to answer the 
question correctly. For example, final-year students scoring at or above 478 were 
likely to provide a correct response to the item asking about the number of defective 
light bulbs, and those scoring at or above 646 were likely to respond correctly to the 
problem about the increase in volume of the cube-shaped carton. Considering that 
the international average on the scale was 500, students achieving at about the level of 
the international average were unlikely to have answered the latter item correctly. 
These results, however, varied dramatically by country. For example, students in 
higher-performing countries were much more likely than students in lower-performing 
countries to answer correctly all but the most difficult of the mathematics literacy 
questions. In general, the most difficult questions asked students to apply their 
knowledge of mathematics to particular real-world situations or to use multiple 
pieces of information in responding. 




The three-digit item label shown in the lower right corner of the box locating each example item on the item 
difficulty map refers to the original item identification number used in the student test booklets. 
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Mathematics Literacy 



Percent Correct for Example Item 1 

Final Year of Secondary Sclhool* 





: ,N~‘ 

-Rendeht-H 


1®!-' -1- 


: 6’x'ample 1 1 ' ' 7 'T 1; T 


; Country 


Correct 

' >/r r ' *vW : ; 




\ -rf i : T^Qd^.|^CtLo.n.« : V , 


2 Cyprus 


66 (3.2) 


48% 




Czech Republic 


61 (5.1) 


78% 


If there are 300 calories in 100 grams of a certain food, how many calories are 


Hungary 


59 (1.3) 


65% 


there in a 30 gram portion of that food? 


1 Lithuania 


67 (2.6) 


43% 




f New Zealand 


75 (2.9) 


70% 


® 90 


2 Russian Federation 


71 (2.4) 


48% 


B. 100 


Sweden 


74 (1.6) 


71% 


C. 900 


Switzerland 


79 (1 .8) 


82% 


Countries Not Satisfying Guidelines for Sample 


D. 1000 


Participation Rates (See Appendix B for Details): 


E. 9000 


Australia 


71 (2.9) 


68% 




2 Austria 


78 (2.2) 


76% 




Canada 


73 (2.3) 


70% 




France 


80 (2.0) 


84% 




Iceland 


75(1.5) 


55% 




1 Italy 


71 (2.3) 


52% 




Norway 


72(1.4) 


84% 




United States 


68(1.3) 


63% 




1 Countries with Unapproved Student Sampling 1 




| (See Appendix B for Details): 






1 Germany 


1 74 (2.3) | 


| 75% 




Countries With Unapproved Sampling Procedures and 
Low Participation Rates (See Appendix B for Details): 




Denmark 


75(1.5) 


58% 




2 Netherlands 


84(1.5) 


78% 




Slovenia 


75 (2.4) 


88% 




South Africa 


45 (2.5) 


49% 




International Average 
Percent Correct 


71 (0.5) 







SOURCE: 1EA Third International Mathematics and Science Study (TIMSS), 1995-96. 



* See Appendix A for characteristics of the students sampled. 

t Met guidelines for sample participation rates only after replacement schools were included (see Appendix B for details). 

1 National Desired Population does not cover all of International Desired Population (see Table B.4). 

2 National Defined Population covers less than 90 percent of National Desired Population (see Table B.4). 

( ) Standard errors appear in parentheses. Because results are rounded to the nearest whole number, some totals may appear inconsistent. 
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Table 3.2 



Mathematics Literacy 



Percent Correct for Example Item 2 
Final Year of Secondary School* 



Country 


Percent 

Correct 


TCI 


Example 2 






Number of defective light bulbs. 


2 Cyprus 


52 (3.5) 


48% 




Czech Republic 


63 (2.8) 


78% 


From a batch of 3000 light bulbs, 100 were selected at random and tested. If 5 


Hungary 


52 (1 .4) 


65% 


of the light bulbs in the sample were found to be defective, how many defec- 
tive light bulbs would be expected in the entire batch? 


1 Lithuania 


54 (3.0) 


43% 




1 New Zealand 


77(1.8) 


70% 


A. 15 


2 Russian Federation 


57 (2.4) 


48% 


B. 60 

150 


Sweden 


77(1.3) 


71% 


Switzerland 


72 (2.3) 


82% 


Countries Not Satisfying Guidelines for Sample 
Participation Rates (See Appendix B for Details): 


D. 300 

E. 600 


Australia 


74 (2.4) 


68% 


2 Austria 


73 (2.2) 


76% 




Canada 


70 (2.6) 


70% 




France 


73(1.9) 


84% 




Iceland 


68 (1.5) 


55% 




1 Italy 


60 (2.6) 


52% 




Norway 


67 (1 .5) 


84% 




United States 


62 (1 .8) 


63% 




1 Countries with Unapproved Student Sampling 




|(See Appendix B for Details): 






1 Germany 


| 66 (3.3) ! 


| 75% 




Countries With Unapproved Sampling Procedures and 
Low Participation Rates (See Appendix B for Details): 




Denmark 


73(1.5) 


58% 




2 Netherlands 


85(1.5) 


78% 




Slovenia 


77 (2.4) 


88% 




South Africa 


34 (2.3) 


49% 




International Average 
Percent Correct 


66 (0.5) 







SOURCE: IEA Third International Mathematics and Science Study (TIMSS), 1995-96. 



* See Appendix A for characteristics of the students sampled. 

* Met guidelines for sample participation rates only after replacement schools were included (see Appendix B for details). 

1 National Desired Population does not cover all of International Desired Population (see Table B.4). 

2 National Defined Population covers less than 90 percent of National Desired Population (see Table B.4). 

( ) Standard errors appear in parentheses. Because results are rounded to the nearest whole number, some totals may appear inconsistent. 








